Implementing BCS connector based on enumerator methods

Article
12/14/2011

Business Connectivity Services (BCS) in SharePoint 2010 is all about accessing data external to SharePoint. BCS provides read/write access to external data from line-of-business (LOB) systems, Web services, databases, and other external systems within Microsoft SharePoint 2010 and Microsoft Office 2010 applications. BCS enhances the SharePoint platform’s capabilities with out-of-box features, services and tools that streamline development of solutions with deep integration of external data and services. Details can be found here.

The core function of BDC is to provide connectivity support to the following types of external systems:

Databases
Web/WCF services
Microsoft .NET Framework connectivity assemblies
Custom data sources

There are scenarios when we wish to do custom implementation to achieve the desired processing. This can be achieved by developing Visual Studio based code solutions. We add up code to do the required processing for implementing List\Item operations. These solutions work great and provide a lot of flexibility.

However, in scenarios where BCS is used to pull data and index in SharePoint for implementing search driven applications, we need constant updation of data from the external system. In such systems we need incremental update to work to ensure CRUD operations. In such a scenario we need to create solutions that implement ChangedIDEnumerator\DeletedIDEnumerator apart from general List (IDEnumerator), Item (ReadItem) operations.

IDEnumerator - Used for list operation

ChangedIDEnumerator - Identifies the modified items

DeletedIDEnumerator - Identifies the deleted items

ReadItem - used for Item operation

apart from the above we may need to implement streamaccessor method for ingesting streams for indexing like documents.

Here, we will have a look on the methods and the mapping in model file.

IDEnumerator

Syntax: public static IEnumerable<EntityType> IDEnumerator()

Returns an enumerator of EntityType. This will have all the items that need to be indexed. This can be infered as the finder method.

Model file mapping:

             <Method IsStatic="false" Name="IDEnumerator">
              <Parameters>
                <Parameter Direction="Return" Name="returnParameter">
                  <TypeDescriptor TypeName="System.Collections.Generic.IEnumerable`1[EntityType Class details]" IsCollection="true" Name="ListName">
                    <TypeDescriptors>
                      ...
                    </TypeDescriptors>
                  </TypeDescriptor>
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Type="IdEnumerator" ReturnParameterName="returnParameter" Default="true" Name="EnumeratorName" DefaultDisplayName="DisplayName">
                  <Properties>
                    <Property Name="RootFinder" Type="System.String">x</Property>                    
                  </Properties>
                </MethodInstance>
              </MethodInstances>
            </Method>

ChangedIDEnumerator

Syntax: public static IEnumerable<EntityType> ChangedIDEnumerator(DateTime LastRunDate)

Returns an enumerator of EntityType. it contains the modified items. LastRunDate indicates time that will be used as reference for this crawl for finding the modified items. We can map this value or can use another parameter (configuration parameter, or reading from external ways) for getting the reference.

Model file mapping:

          <Method IsStatic="false" Name="ChangedIDEnumerator">
              <FilterDescriptors>
                <FilterDescriptor Name="LastRunDate" Type="InputOutput">
                  <Properties>
                    <Property Name="SynchronizationCookie" Type="System.String">LastModifiedDate</Property>
                  </Properties>
                </FilterDescriptor>
                <FilterDescriptor Name="FilterDescriptor" Type="Timestamp" />
              </FilterDescriptors>
              <Parameters>
                <Parameter Name="@LastRunDate" Direction="InOut">
                  <TypeDescriptor Name="LastRunDateTypeDescriptor" TypeName="System.DateTime" AssociatedFilter="LastRunDate">
                    <Interpretation>
                      <NormalizeDateTime LobDateTimeMode="Local" />
                    </Interpretation>
                  </TypeDescriptor>
                </Parameter>
                <Parameter Direction="Return" Name="returnParameter">
                  <TypeDescriptor TypeName="System.Collections.Generic.IEnumerable`1[EntityType Class details]" IsCollection="true" Name="ListName">
                    ...
                  </TypeDescriptor>
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Type="ChangedIdEnumerator" ReturnParameterName="returnParameter" Default="true" Name="OperationName" DefaultDisplayName="DisplayName" />
              </MethodInstances>
            </Method>

 DeletedIDEnumerator

Syntax: public static IEnumerable<EntityType> DeletedIDEnumerator(DateTime LastRunDate)

Returns an enumerator of EntityType. it contains the deleted items. LastRunDate indicates time that will be used as reference for this crawl for finding the deleted items. We can map this value or can use another parameter (configuration parameter, or reading from external ways) for getting the reference.

         <Method IsStatic="false" Name="DeletedIDEnumerator">
              <FilterDescriptors>
                <FilterDescriptor Name="LastRunDate" Type="InputOutput">
                  <Properties>
                    <Property Name="SynchronizationCookie" Type="System.String">LastModifiedDate</Property>
                  </Properties>
                </FilterDescriptor>
                <FilterDescriptor Name="FilterDescriptor" Type="Timestamp" />
              </FilterDescriptors>
              <Parameters>
                <Parameter Name="@LastRunDate" Direction="InOut">
                  <TypeDescriptor Name="LastRunDateTypeDescriptor" TypeName="System.DateTime" AssociatedFilter="LastRunDate">
                    <Interpretation>
                      <NormalizeDateTime LobDateTimeMode="Local" />
                    </Interpretation>
                  </TypeDescriptor>
                </Parameter>
                <Parameter Direction="Return" Name="returnParameter">
                  <TypeDescriptor TypeName="System.Collections.Generic.IEnumerable`1[EntityType class details]" IsCollection="true" Name="ListName">
                    <TypeDescriptors>
                      ...
                    </TypeDescriptors>
                  </TypeDescriptor>
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Type="DeletedIdEnumerator" ReturnParameterName="returnParameter" Default="true" Name="OperationName" DefaultDisplayName="DisplayName" />
              </MethodInstances>
            </Method>

ReadItem

Syntax: public static Entity ReadItem(string ID)

Returns type entity. it contains the detail of an item. ID is the unique identifier for the item.

Mapping in model:

        <Method IsStatic="false" Name="ReadItem">
              <Parameters>
                <Parameter Name="ID" Direction="In">
                  <TypeDescriptor Name="ID" TypeName="System.String" IdentifierEntityName="ENTITY" IdentifierEntityNamespace="NAMESPACE" IdentifierName="ID">
                  </TypeDescriptor>
                </Parameter>
                <Parameter Name="returnParameter" Direction="Return">
                  ...
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Name="ReadItem" Type="SpecificFinder" ReturnParameterName="returnParameter" Default="true" DefaultDisplayName="DisplayName">
                  <Properties>
                    <Property Name="LastDesignedOfficeItemType" Type="System.String">None</Property>
                    <Property Name="WindowsSecurityDescriptorField" Type="System.String">SecurityDescriptor</Property>
                  </Properties>
                </MethodInstance>
              </MethodInstances>
            </Method>

 Here, SecurityDescriptor is used for implementing security at record level.

 StreamAccessor

 Syntax: public Stream GetStream(string ID)

 GetStream can be used if we intend to ingest documents in the crawl process. ID is the unique identifier.

 Mapping in Model:

           <Method IsStatic="false" Name="GetStream">
              <Parameters>
                <Parameter Name="ID" Direction="In">
                  <TypeDescriptor Name="ID" TypeName="System.String" IdentifierEntityName="ENTITY" IdentifierEntityNamespace="NAMESPACE" IdentifierName="ID" />
                </Parameter>
                <Parameter Name="returnParameter" Direction="Return">
                  <TypeDescriptor Name="returnParameterTypeDescriptor" TypeName="System.IO.Stream" />
                </Parameter>
              </Parameters>
              <MethodInstances>
                <MethodInstance Name="MethodName" Type="StreamAccessor" ReturnParameterName="returnParameter">
                  <Properties>
                   ...
                  </Properties>
                </MethodInstance>
              </MethodInstances>
            </Method>

 More on custom connectors can be found here.

 Another great post on BCS can be found here.

Implementing BCS connector based on enumerator methods

Additional resources