Monthly Archives: July 2008

Documentum 6.5 with services in mind

You may have seen this week’s press release (PR) concerning the launch of the EMC Documentum 6.5 platform. Given that this blog is referenced in the PR’s sidebar, I thought it would be appropriate for me to offer my perspective on this release.

When the Documentum Foundation Services (DFS) team first embarked on its mission to better service-orient the Documentum platform, I suggested that, over time, the very focus of our daily work–services and common service infrastructure–would become less “talked about.” I suggested that services alone deliver less value than services in the context of processes and processes in the context of solutions.

So, ultimately EMC has to deliver relevant, robust solutions to market much more than just web services. This is exactly the focus of the EMC PR–and each solution discussed is built using Enterprise Content Services (ECS).

Take CenterStage (aka “Magellan”), Media WorkSpace, and My Documentum as D6.5 solution examples. One user experience differs fairly significantly from another user experience (e.g. CenterStage Essentials versus Media WorkSpace). Looking under the covers, one implementation (i.e. presentation layer and application behavior) also differs from another implementation. However, common business logic is provided to each solution via a consistent set of services (e.g. Object, Query, Schema, Access Control, etc.).

Each solution is worth its own coverage, but I will allow solution teams to accomplish this (e.g. content like this).

Instead, I’d like to focus on the progress achieved in the D6.5 release where services and common service infrastructure are concerned:

1. The number of out-the-box Enterprise Content Services (ECS) in DFS has more than doubled in 6.5, from the six to 13. The total number of ECS across D6.5 has grown nearly fourfold, from six to 23.

  • Six original services with DFS 6.0: Object (enhanced in 6SP1 to support external objects; enhanced again in 6.5 to support aspects), Query, Schema, Search (enhanced functionality in 6.5 for clusters), Version Control (enhanced in 6.5 to support aspects), Workflow
  • Seven new services with DFS 6.5: Access Control, Analytics, Comment, Lifecycle, Query Store, Task Management (WS-HumanTask), Virtual Document
  • Plus 10 additional, new services beyond (i.e. not packaged with) DFS 6.5:
    Content Delivery, Electronic Signature, ERP Integration (SAP), Federated Proxy, Formal Record, Physical Records Library, (Records Management) Policy, (Content Transformation) Profile, Retention Markup, (Content) Transformation

Bottom line #1: there are many more services with D6.5 from which to compose content-centric applications.

2. Common service infrastructure has been significantly enhanced at the same time. For example:

  • Contract first (or WSDL first) development of services is supported in addition to code first development.
  • Design time discovery of services is supported via a new service catalog or an existing UDDI v2 compliant service registry. Classification of service occurs via the concept of catalogs and categories, by default.
  • Modularity has increased at the binary (e.g. JAR file) level to promote better composition (i.e. only take what you need).
  • Single sign-on (SSO) support has been improved.

Bottom line #2: you have more options now to provide, discover and consume services.

Certainly there is a lot more to each bullet point above, and I plan to drill down into several points in future posts.

…and I didn’t even use the term “Web 2.0″…until now. :-)

DFS Object service consumer #2b

In our previous conversation, we left with a missing header to resolve. This post will resolve this issue and wrap-up the sample DFS Object service consumer based on direct proxy via Visual Studio/WCF.

First, we need to introduce a new internal class, ServiceContextHeader, as follows:

internal class ServiceContextHeader : MessageHeader
{
    private string repositoryName = null;
    private string userName = null;
    private string password = null;

    public ServiceContextHeader(string repositoryName, string userName, string password)
    {
        this.repositoryName = repositoryName;
        this.userName = userName;
        this.password = password;
    }

    protected override void OnWriteHeaderContents(XmlDictionaryWriter writer, MessageVersion messageVersion)
    {
        // Please note that there can be more to a ServiceContext instance than
        // what this intentionally simple example specifies. This message header
        // implementation doesn't cover all aspects of ServiceContext.

        // In order to proxy ServiceContext, you must first establish a service
        // reference to the DFS runtime Context Registry service.
        ServiceContext serviceContext = new ServiceContext();
        RepositoryIdentity repositoryIdentity = new RepositoryIdentity();
        repositoryIdentity.repositoryName = repositoryName;
        repositoryIdentity.userName = userName;
        repositoryIdentity.password = password;
        serviceContext.Identities = new Identity[1];
        serviceContext.Identities[0] = repositoryIdentity;

        // We haven't communicated with the DFS runtime Context Registry service;
        // so we don't have a token to "write," nor are we specifiying locale in 
        // our ServiceContext instance.

        StringWriter sw = new StringWriter();
        XmlSerializer xs = new XmlSerializer(typeof(ServiceContext));
        xs.Serialize(sw, serviceContext);
        String xml = sw.ToString();
        sw.Close();

        XmlDocument document = new XmlDocument();
        document.LoadXml(xml);

        foreach (XmlNode node in document.LastChild.ChildNodes)
        {
            XmlElement element = (XmlElement)node;
            XmlReader reader = new XmlNodeReader(element);
            writer.WriteNode(reader, true);
        }
    }

    public override string Name
    {
        get { return "ServiceContext"; }
    }

    public override string Namespace
    {
        get { return "http://context.core.datamodel.fs.documentum.emc.com/"; }
    }
}

Next, we need to “inject” this class into our application’s request message processing. So, we need wrap our original call to ObjectService.get() as follows:

DataPackage dataPackage = null;
using (OperationContextScope scope = new OperationContextScope(objectService.InnerChannel))
{
    OperationContext.Current.OutgoingMessageHeaders.Add(new ServiceContextHeader(repositoryName, userName, password));

    dataPackage = objectService.get(objectIdentitySet, operationOptions);
}

In a less contrived context (i.e. your production application), there are other approaches to be considered, and I recommend that you at least read Nick Allen’s post on the subject.

Now, let’s re-run the modified example and examine the SOAP request message:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
        <ServiceContext xmlns="http://context.core.datamodel.fs.documentum.emc.com/">
            <Identities xsi:type="RepositoryIdentity" userName="_USER_" password="_PWD_" repositoryName="_DOCBASE_" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
        </ServiceContext>
    </s:Header>
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <get xmlns="http://core.services.fs.documentum.emc.com/">
            <forObjects isInternal="false" xmlns="">
                <Identities repositoryName="_DOCBASE_" valueType="OBJECT_ID" xmlns="http://core.datamodel.fs.documentum.emc.com/">
                    <ObjectId id="09003023800037c6"/>
                </Identities>
            </forObjects>
            <options xmlns="">
                <Properties isInternal="false" xmlns="http://core.datamodel.fs.documentum.emc.com/"/>
                <Profiles xsi:type="q1:PropertyProfile" isProcessIncludedUnknown="false" filterMode="SPECIFIED_BY_INCLUDE" xmlns="http://core.datamodel.fs.documentum.emc.com/" xmlns:q1="http://profiles.core.datamodel.fs.documentum.emc.com/">
                    <q1:IncludeProperties>object_name</q1:IncludeProperties>
                </Profiles>
            </options>
        </get>
    </s:Body>
</s:Envelope>

Much better! And, as important, the dialog reveals the same object name as the same example based on the DFS SDK .NET assemblies.

You can download the Visual Studio 2008-based solution for today’s (fully resolved) sample here.

I hope that this series of posts on how the EMC Documentum platform, and especially DFS, supports Microsoft-oriented developers has been helpful.

Cheers… :-)

Update 8/9/2008: There is actually one additional change that is required to properly retrieve the object’s name for presentation on the form. In the previous post, after calling ObjectService.get() and finding the “object_name” property, our form label was set as follows:

    lblObjName.Text = property.ToString();

However, this needs to change in order to avoid prefacing the string value with ToString() content:

    StringProperty s = property as StringProperty;
    lblObjName.Text = s.Value;

This sample has been updated to include this change. Kudos to John Sweeney for the assist.

OK, back to the Olympics… :-)

DFS Object service consumer #2a

Rather than leverage the DFS SDK-based .NET assemblies to consume DFS services, some may prefer to leverage Visual Studio’s ability to proxy WSDL/XSD directly and choose instead to involve the Windows Communication Foundation (WCF) designer via Add Service Reference… In this post, I will discuss just such a sample application.

As in the previous example, we’ll build the same Windows Forms application to accomplish the same task (i.e. object name retrieval given object id); however we’ll ignore the .NET productivity layer and proxy the Object service contract directly using Visual Studio and WCF.

After launching Visual Studio, choosing File | New | Project… and creating a new Windows Forms Application project called ObjectServiceConsumer, go to the Solution Explorer, select the project node, right-click and choose Add Service Reference…

Enter the address for the DFS Object service instance to proxy (e.g. the same instance used in the previous example’s browser-based accessibility test). Click Go, and you should see something similar to the dialog captured above. Set the namespace for this reference and choose OK.

Return back to Visual Studio, double-click Form1.cs, after building the same dialog as in the previous example, double-click on the Retrieve button. This will tell Visual Studio to transition from (Windows Forms) designer mode to code editing mode.

Implement your button click handler as follows:

string repositoryName = txtbxRepoName.Text;
string userName = txtbxUserName.Text;
string password = txtbxPwd.Text;
string objId = txtbxObjId.Text; // e.g. a document id of 09123456789abcde

try
{
    ObjectId objectId = new ObjectId();
    objectId.id = objId;
    ObjectIdentity objectIdentity = new ObjectIdentity();
    objectIdentity.Item = objectId;
    objectIdentity.repositoryName = repositoryName;
    ObjectIdentitySet objectIdentitySet = new ObjectIdentitySet();
    objectIdentitySet.Identities = new ObjectIdentity[1];
    objectIdentitySet.Identities[0] = objectIdentity;

    PropertyProfile propertyProfile = new PropertyProfile();
    propertyProfile.filterMode = PropertyFilterMode.SPECIFIED_BY_INCLUDE;
    string[] includeProperties = {"object_name"};
    propertyProfile.IncludeProperties = includeProperties;
    OperationOptions operationOptions = new OperationOptions();
    operationOptions.Profiles = new Profile[1];
    operationOptions.Profiles[0] = propertyProfile;

    DataPackage dataPackage = objectService.get(objectIdentitySet, operationOptions);

    Property[] properties = dataPackage.DataObjects[0].Properties.Properties;
    foreach (Property property in properties)
    {
        if (property.name.Equals("object_name"))
        {
            lblObjName.Text = property.ToString();
        }
    }

    Console.WriteLine("Successfully retrieved object name for object id" + "'" + objId + "': " + lblObjName.Text);
}
catch (Exception ex)
{
    lblObjName.Text = "<error>";
    Console.WriteLine(ex.StackTrace);
    Console.WriteLine("Failed to retrieve object name with exception " + ex.Message);
}
finally
{
    Console.WriteLine("Your cleanup logic goes here.");
}

A few points concerning the above code:

  • There is a private field on the Form class, objectService, that is of type ObjectServicePortClient (i.e. from the WCF-generated proxy code) and is set to null initially.
  • In the Form class constructor, after the standard Windows Forms InitializeComponent(), objectService is set as follows: new ObjectServicePortClient(“ObjectServicePort”);
  • “ObjectServicePort” is a named binding in the WCF-generated proxy code (i.e. declared within app.config).
  • As in the previous example, I’m using a pre-release version of DFS 6.5 for this example, but I’m not using anything that isn’t available in DFS 6.0 SP1–this is a very simple example by design.
  • You should notice some cosmetic differences between the code above and the code here (e.g. more verbose, fewer conveniences, etc.).

You should now be able to build your application and run it. When you enter a valid object id for the specified repository connection, you should see the object name replace “<tbd>” on the dialog.

But, you don’t. Why?

Well, first of all, if you are running your application server with a console window for output message capture, you likely saw the following message:

. . . com.emc.documentum.fs.rt.SerializableException: Authorization failed, could not find identities in service context with token “temporary/127
.0.0.1-1216079837407–1167916486618885387″
        at com.emc.documentum.fs.services.core.ObjectServiceWebService.get(ObjectServiceWebService.java:268) . . .

Looking at the code above, it’s hopefully clear that the variables userName and password aren’t employed. (If you’re using the JetBrains ReSharper add-in for Visual Studio, the IDE (plugin) actually visually indicates this condition.)

So, let’s do some detective work.

Using a web debugging proxy (e.g. Charles), re-run the above code (i.e. with port forwarding in place). After valid data entry and clicking the Retrieve button, you should see an entry in your web debugging proxy session. Examine the SOAP request message (i.e. “pretty printed” for readability’s sake):

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <get xmlns="http://core.services.fs.documentum.emc.com/">
            <forObjects isInternal="false" xmlns="">
                <Identities repositoryName="_DOCBASE_" xmlns="http://core.datamodel.fs.documentum.emc.com/">
                    <ObjectId id="09003023800024ec"/>
                </Identities>
            </forObjects>
            <options xmlns="">
                <Profiles xsi:type="q1:PropertyProfile" isProcessIncludedUnknown="false" xmlns="http://core.datamodel.fs.documentum.emc.com/" xmlns:q1="http://profiles.core.datamodel.fs.documentum.emc.com/">
                    <q1:IncludeProperties>object_name</q1:IncludeProperties>
                </Profiles>
            </options>
        </get>
    </s:Body>
</s:Envelope>

Next, go back to the previous example, and re-run it to capture its SOAP request message. Before you hit Run, though, comment out the following line of code:

context = contextFactory.Register(context);

(We’ll talk more about this in a future post.)

The DFS SDK .NET assemblies-based application issues the following SOAP request message:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
        <ServiceContext token="temporary/127.0.0.1-1216195928765-692634301" xmlns="http://context.core.datamodel.fs.documentum.emc.com/">
            <Identities xsi:type="RepositoryIdentity" userName="_USER_" password="_PWD_" repositoryName="_DOCBASE_" domain="" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"/>
            <RuntimeProperties/>
        </ServiceContext>
    </s:Header>
    <s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <get xmlns="http://core.services.fs.documentum.emc.com/">
            <forObjects isInternal="false" xmlns="">
                <Identities repositoryName="_DOCBASE_" valueType="OBJECT_ID" xmlns="http://core.datamodel.fs.documentum.emc.com/">
                    <ObjectId id="09003023800037c6"/>
                </Identities>
            </forObjects>
            <options xmlns="">
                <Properties xmlns="http://core.datamodel.fs.documentum.emc.com/"/>
                <Profiles xsi:type="q1:PropertyProfile" filterMode="SPECIFIED_BY_INCLUDE" xmlns="http://core.datamodel.fs.documentum.emc.com/" xmlns:q1="http://profiles.core.datamodel.fs.documentum.emc.com/">
                    <q1:IncludeProperties>object_name</q1:IncludeProperties>
                </Profiles>
            </options>
        </get>
    </s:Body>
</s:Envelope>

Yep, there’s definitely a missing header in today’s sample, as coded above. :-)

It’s also worth noting a few things before continuing:

  • The code above requires SSL to avoid cleartext data being sent over the wire. Of course, SSL–and replacing http://… with https://…–only provides transport-level security that is point-to-point in nature (i.e. compared with message-level security that is end-to-end in nature).
  • valueType=”OBJECT_ID” – this is missing on the <Identities> element in today’s example but is present in the previous example
  • <Properties xmlns=”http://core.datamodel.fs.documentum.emc.com/”/> – this is missing in today’s example but is present in the previous example
  • filterMode=”SPECIFIED_BY_INCLUDE” – this is missing on the <Profiles> element in today’s example but is present in the previous example
  • isProcessIncludedUnknown=”false” – this is present in today’s example but is missing in the previous example

Before we solve the case of the missing header, let’s resolve the differences just listed:

...
ObjectIdentity objectIdentity = new ObjectIdentity();
objectIdentity.valueType = ObjectIdentityType.OBJECT_ID; //FIX
objectIdentity.valueTypeSpecified = true; //FIX
...
propertyProfile.filterMode = PropertyFilterMode.SPECIFIED_BY_INCLUDE;
propertyProfile.filterModeSpecified = true; //FIX
...
OperationOptions operationOptions = new OperationOptions();
...
operationOptions.Properties = new PropertySet(); //FIX
...

As for the isProcessIncludedUnknown-based difference aforementioned, a quick read of the reference-level documentation for PropertyProfile.setProcessIncludedUnknown() is helpful: “If false, ignore any property in the includeProperties list that is not a property of the repository type. If true, throw an exception if such a property is specified in the includeProperties list. Default value is false.” That is, the WCF-generated proxy makes this default explicity in its SOAP messages; so, although the two XML messages have this difference, they are functionally equivalent.

Next: Leveraging WCF extensibility to add the missing header that the DFS Object service expects…

DFS Object service consumer #1

In a previous post, I introduced web service support in the EMC Documentum platform for the Microsoft-oriented developer. (BTW, sorry for the time lapse since that post to this one–busy getting an important set of products ready for on-time release. :-) ) In this post, I will focus on such support as provided with DFS via the .NET assemblies (aka productivity layer) within the DFS SDK.

Let’s build a simple Windows Forms application using Microsoft Visual Studio 2008. The purpose of this application will be to retrieve the object name from a piece of content in a Documentum repository associated with a particular object id:

After launching Visual Studio, choosing File | New | Project… and creating a new Windows Forms Application project called ObjectServiceConsumer_PL, go to the Solution Explorer, select the References node under the project, right-click and choose Add Reference…

Navigate to your DFS SDK installation and select the set of .NET assemblies that support the DFS Object service. For example, I tend to install the SDK directly under my C: drive; so, I navigated to C:\emc-dfs-sdk-6.5\lib\dotnet:

(Yes, I’m using a pre-release version of DFS 6.5 for this example, but I’m not using anything that isn’t available in DFS 6.0 SP1–this is a very simple example by design.)

The result under the References node should be as follows:

If you’re already developing with .NET 3.5, you may notice that a couple of .NET 3.5 assemblies are missing in the above view. That’s because I decided to target .NET 3.0 for this example since I don’t require anything more from Windows Communication Foundation since its first release (i.e. DFS supports WCF “v1″ in .NET 3.0).

At this point, verify that you will be able to access an instance of the DFS Object service from your application. That is, deploy emc-dfs.ear file via the standalone DFS installer or with the Content Server installer. Launch your browser of choice and retrieve DFS Object service WSDL (e.g. http://localhost:8080/services/core/ObjectService?wsdl). You should see the service contract in your browser.

Return back to Visual Studio, double-click Form1.cs, after building the dialog shown above, double-click on the Retrieve button. This will tell Visual Studio to transition from (Windows Forms) designer mode to code editing mode.

Implement your button click handler as follows:

string repositoryName = txtbxRepoName.Text;
string userName = txtbxUserName.Text;
string password = txtbxPwd.Text;
string objId = txtbxObjId.Text; // e.g. a document id of 09123456789abcde

try
{
    ContextFactory contextFactory = ContextFactory.Instance;
    IServiceContext context = contextFactory.NewContext();
    RepositoryIdentity repoId = new RepositoryIdentity(repositoryName,
                                                       userName,
                                                       password,
                                                       "");
    context.AddIdentity(repoId);

    context = contextFactory.Register(context); // Module and ContextRoot from app.config
    ServiceFactory serviceFactory = ServiceFactory.Instance;

    IObjectService objectService =
        serviceFactory.GetRemoteService<IObjectService>(context); // Module and ContextRoot from app.config

    ObjectId objectId = new ObjectId(objId);
    ObjectIdentity objectIdentity = new ObjectIdentity(objectId, repositoryName);
    ObjectIdentitySet objectIdentitySet = new ObjectIdentitySet(objectIdentity);

    PropertyProfile propertyProfile = new PropertyProfile();
    propertyProfile.FilterMode = PropertyFilterMode.SPECIFIED_BY_INCLUDE;
    propertyProfile.IncludeProperties.Add("object_name");

    OperationOptions operationOptions = new OperationOptions();
    operationOptions.Profiles.Add(propertyProfile);

    DataPackage dataPackage = objectService.Get(objectIdentitySet, operationOptions);

    List<Property> properties = dataPackage.DataObjects[0].Properties.Properties;

    if (properties != null)
    {
        properties.ForEach(delegate(Property property)
                               {
                                   if (property.Name.Equals("object_name"))
                                   {
                                       lblObjName.Text = property.ToString();
                                   }
                               });
    }

    Console.WriteLine("Successfully retrieved object name for object id" + "'" + objId + "': " + lblObjName.Text);
}
catch (Exception ex)
{
    lblObjName.Text = "<error>";
    Console.WriteLine(ex.StackTrace);
    Console.WriteLine("Failed to retrieve object name with exception " + ex.Message);
}
finally
{
    Console.WriteLine("Your cleanup logic goes here.");
}

As noted above in code comments, you should add an Application Configuration File item to your project. After doing so, open the new app.config file in Visual Studio, select its contents and paste (overwrite) them with the contents copied from the sample App.config file in the DFS SDK (i.e. C:\emc-dfs-sdk-6.5\etc\config\App.config). Be sure to modify data relative to your running DFS Object service instance (e.g. ModuleInfo element attributes like port).

You should now be able to build your application and run it. When you enter a valid object id for the specified repository connection, you should see the object name replace “<tbd>” on the dialog.

Obviously this is a trivial sample. However, as I mentioned this is be design, since what I want to do next is contrast this sample with the same goal implemented without DFS SDK .NET assemblies (i.e. using just Visual Studio, its WCF designer and DFS WSDL/XSD content).

You can download the Visual Studio 2008-based solution for today’s sample here.

Next: Using DFS WSDL/XSD (sans DFS SDK .NET assemblies) from Visual Studio…