Monthly Archives: June 2008

Data access, Microsoft developers and DAS

DFC PIA, in turn, became the foundation for Documentum ADO.NET Services (DAS), which released with the EMC Documentum 5.3 platform.

DAS integration with Visual Studio

DAS enables Documentum to be a first-class data source for the large number of freely available data-bound controls from Microsoft and even larger number of commercially available third party data-bound controls. If you knew how to bind a data source to a grid or list view control, then Documentum immediately became relevant to your solution building experience.

DAS integration with Visual Studio

Prior to DAS, Documentum offered a set of special purpose widgets with its Developer Studio license. These ActiveX controls worked well in a classic VB context, where business users typically composed application presentation by choosing controls from a visual palette to create dialogs and composed application behavior by double-clicking through to event handling code (e.g. button click, dropdown select, etc.). For their effectiveness though, each control typically had its own API.

DAS integration with Visual Studio

A developer’s time is a precious business resource; therefore, the more productive a developer can become the greater potential for business benefit. So, rather than expect Microsoft-oriented developers to learn Documentum control APIs, it made more sense to embrace ADO.NET and a fairly rich set of free and commercially available data bound controls.

DAS integration with Visual Studio

That is, DAS was about changing the developer message to one of “your .NET expertise is applicable where content-centric application development with Documentum is concerned.” DAS was about making the Documentum platform more relevant to the .NET developer.

Since the transition from client-server (e.g. Desktop) to web-based (e.g. Webtop) applications (i.e. for awhile now), Documentum has recommended that DFC-based business logic belongs in the middle tier and not on the local (end user) machine. When DAS is used in an ASP.NET application, DFC PIA and therefore DFC are indeed running on the IIS server machine–competing GCs and all. However, when DAS is used in a Windows Forms application, DFC winds up on the end user machine, contradicting business logic topology recommendations.

Next: Web services…

DFC, Microsoft developers, .NET and DFC PIA

In early 2002 Microsoft released .NET 1.0, and it was quite clear that .NET would quickly supplant COM as the mainstream runtime for the Microsoft platform.

Fortunately, .NET provided good (bi-directional) interop with COM (e.g. callable wrappers and interop assemblies). Furthermore, software publishers (ISVs) could produce primary interop assemblies (PIAs) to add functional, deployment and administrative value above that produced by simply pointing Visual Studio at a type library.

So, based on customer requirements and demand, the DFC PIA was born and was first released with DFC 5.2.5.

DFC, DJCB and DFC PIA in relationship

Beyond just producing a strong-named assembly using a Documentum private key, DFC PIA included additional performance tuning as well as some convenience features like advertising deprecated APIs, IntelliSense support in Visual Studio via robust XML documentation, and HTML Help content with example usage. By virtue of being a strong-named assembly, subsequent versions of the DFC PIA shipped with a publisher policy file and assembly, allowing customers to redirect their DFC PIA 5.2.5 consumers to a newer version of the PIA.

Both Documentum Desktop and Documentum Application Builder announced end-of-life plans around D5.3; so the internal need for DJCB was coming to a close. This opportunity was leveraged to reconsider current platform support for .NET developers and what would be needed in the future (e.g. web services). Several data points arose that suggested the need to reset interoperability as follows:

  • Java and .NET are both managed language platforms. A key management facility in both is known as garbage collection. A garbage collector (GC) is responsible for reclaiming resources based on object reference state and other criteria. The Java GC and the .NET GC don’t recognize each other; yet both GC’s can be resident in the same process space. Given this situation, the potential for resource exhaustion due to improper object reference coding arises. In .NET, all COM objects including objects implementing COM interfaces must be released using Marshal.ReleaseComObject or Marshal.FinalReleaseComObject; otherwise, the underlying resources cannot be reclaimed by a GC.
    • Marshal.ReleaseComObject/Marshal.FinalReleaseComObject is low-level “plumbing” code, and as such is typically not the primary domain concern of the business developer.
    • Recall that the DFC .NET interface is the projection of a DFC COM interface, which is, in turn, a projection of a DFC Java interface. The implementation of this interface projection is shared amongst a Java archive (JAR file), a Java-COM bridge DLL and an in-process assembly (PIA). All of this, again, is low-level plumbing that the business developer would rather not be burdened with; however, appropriate Marshal.ReleaseComObject/Marshal.FinalReleaseComObject calls have been delegated unto them.
  • DJCB (and therefore DFC PIA) is about core DFC, not about DFC customizations via BOF (i.e. type-based or service-based business objects–TBOs or SBOs). So, to get similar .NET interop for a custom business object, you would have to re-apply Java-COM bridge technology to that API–assuming that it’s well-formed for .NET interop–and then produce a PIA for the custom business object using the bridge’s COM type library–all low-level plumbing.
    • “Java-COM bridge + PIA” doesn’t scale as well as web services in terms of integration cost. DFS supports web services natively and was designed from the very beginning with SOA in mind.
  • COM has a hard limit of no more than 255 methods per interface. Most object-oriented systems may not encounter this constraint; however, DFC does with respect to sysobject-derived interfaces. That is, DFC 5.3 was OK; however, DFC 6.0 and thereafter surpassed this upper limit. So DJCB literally cannot be revised to accommodate the DFC 6.0 interface, again due to COM-based limitations. Since DJCB is so-constrained, DFC PIA is also limited.
    • Arguably, this highlights the value of Aspects in DFC/BOF.
    • This also highlights why DFS is absolutely not a web service projection of the DFC API. Rather DFS is a coarser-grained API that is less chatty and less overwhelming to understand (e.g. where total number of API methods is concerned).
    • In other words, DFC 6.0 was not developed with DJCB/PIA exposure, unlike DFC 5.3 and 5.3 service packs.
  • There are aspects of the DFC Java API that are not interoperable. Perhaps the most requested feature request concerns byte stream support. That is, DFC specifies ByteArrayInputStream, but .NET doesn’t recognize this Java type.
    • DFS supports byte streaming use cases as follows:
      1. The application invokes DFS from the middle tier, gets a byte array–DFS doesn’t write to disk–which is then returned by the application to the end user.
      2. The application invokes DFS from the middle tier, gets a URL which is then which is then passed by the application to the end user. The end user can stream the content using the URL.
      3. The application invokes DFS from the middle tier. UCF client is deployed on the end user box. Content is streamed by UCF from the CS machine directly to the end user.
    • Also ByteArrayInputStream can be impractical even for Java-based solutions, since the type puts all data in memory (e.g. large file transfer concerns).

Next: Data access (ADO.NET)…

DFC, Microsoft developers, COM and DJCB

Prior to DFS, the primary solution-building, business logic-focused EMC Documentum platform API was Documentum Foundation Classes (DFC).

When I joined Documentum (now EMC) over ten years ago, DFC was under development as part of the significant 4i release (aka “Project Piper”). Back then Microsoft shipped a leading JVM and had one of the best IDEs for Java in Visual J++, according some of my then hardcore Java programming friends. The MSJVM intrinsically supported–among other “features” like delegates that led to the Java cold war (lawsuit) between Sun and Microsoft–interoperability with Microsoft’s major runtime play at that time called COM.

The aforementioned cold war led to Microsoft dropping development and support of MSJVM (and led to .NET, J# and JLCA), ending its Java support with the then-outdated 1.1.4 release from Sun. This meant that in order to support COM-based interop from Java you had to either build or buy. And a build-versus-buy decision was especially relevant to Documentum at the time given that the most widely used client/application was Desktop (aka Desktop Client, or just DTC–later called Desktop Development Kit), not to mention the MFC-based Developer Studio (later called Documentum Application Builder (DAB), which is now replaced by Composer). We opted for build, since there was no buy option available then that met our requirements, and the Documentum Java-COM Bridge (DJCB) was born and was first released with DFC 5.0.

DJCB enabled us to project DFC Java types as DFC COM types via COM type libraries and IDL. It also enabled Documentum to adopt newer JVM technology (e.g. Java 2 collections, Sun Hotspot JVM performance improvements, etc.). Furthermore, in the strong Documentum tradition of supporting existing IT infrastructure such as operating systems, relational databases and application servers, whose JVM was running became less of an issue (e.g. Sun, IBM, BEA (now Oracle), etc.).

Here is perhaps an interesting portion of the original DJCB function specification dated 9/5/2001, which I authored with Victor Spivak, describing why DJCB was pursued:

Why does Documentum need any bridge?

Documentum Foundation Classes, or DFC as it is more commonly referenced, is Documentum’s platform-independent business logic layer. A key benefit of the Java language is platform independence, or “Write Once, Run Anywhere” (i.e. anywhere that supports the Java platform); therefore, DFC is implemented in Java.

Documentum is a platform provider, and as a platform company it supports the two dominant platforms of today: Windows and Java. A significant amount of Windows development is COM-based, and from a developer’s point of view, Documentum APIs on Windows are COM interface methods, etc.

When the DFC first decided to use Java as its implementation language, the Java platform was alive and well on Windows. In fact, Microsoft provided a fairly transparent bridge between COM and Java through its Java Virtual Machine (JVM) very soon after Java’s formal introduction to the general software development community. This bridge benefited Documentum a great deal; through a single implementation with minimal wrapping, Documentum could provide a business logic layer (DFC) that was consumable on both Windows and Java environments. … 

Why does Documentum need a new bridge?

The settlement of the lawsuit between Sun and Microsoft has effectively frozen Microsoft’s use of Java in Windows at version 1.1.4. For its own use, Sun has already completed its end-of-life (EOL) of 1.1.4–only 1.1.8 remains available and supported in the Java 1.1 platform. Since Microsoft’s Java-COM bridge is built into its JVM, this bridge is also frozen at version 1.1.4. However, Java hasn’t stopped growing or maturing.

The Java platform tripled in size between Java 1.1 and Java 1.2, and the Java 2 platform  has continued to grow beyond version 1.2. The current release is 1.3 and version 1.4 is available today as a beta product. …

Clearly the Java platform of today is significantly different than version 1.1.4 of the Java platform.

Developers of a platform adopt the models of that platform. Java 2 offers models that are unsupported by DFC today because DFC is constrained by obsolete bridge technology. That is, Java developers using DFC may be forced to use proprietary workarounds to standard solutions that Java 2 provides (e.g. Java Collection Framework).

Java developers expect to leverage and interact with the facilities of the Java 2 platform (i.e. version 1.2 or later). Their encounter with the current DFC programming model may less than desirable as long as DFC is constrained to a much older Java platform and development model. Critical aspects of the Java platform such as security, extensibility, collections, serialization, multithreading, have undergone significant change. DFC will benefit from these changes but it must first be able to depend on more current Java environments–something it cannot do under the current Microsoft–provided bridge.

It’s not necessary to maintain a lowest common denominator approach any longer in DFC. A new bridge can and should be developed that enables DFC to leverage the Java 2 platform for its own benefit and for the benefit of all Java developers who rely on Documentum as part of their business solutions. Providing more mature facilities in DFC to Java developers should not come at the expense of COM developers [i.e. C++/”Classic VB” developers that use COM to achieve binary compatibility]. Existing DFC APIs continue to appear as COM objects with DJCB as they have with the Microsoft bridge. …

Looking at the need for a new bridge another way, Documentum’s customers as well as Documentum itself suffers from the Sun-Microsoft lawsuit and settlement. Documentum cannot afford to drop its support of COM in the business logic layer. Documentum can also not afford to hand-tie Java business logic to an obsolete version of Java. Developers approaching the Documentum platform from a Java perspective expect our Java APIs to be Java 2-like. DJCB is about healing the wounds caused by Sun and Microsoft to our developer base… DJCB maintains a COM developer’s expectations of DFC leverage, while enabling DFC to meet the expectations of Java developers leverage Documentum’s business logic layer implemented in Java.

Next: DFC PIA…

Resetting the interop button

Executive Summary

Since this is a longer post (actually a set of posts), allow me to summarize the main points from the get-go as follows:

  • DFC Primary Interop Assembly (PIA) and the underlying Documentum Java-COM Bridge (DJCB) support the DFC 5.3 API. That is, interface changes in DFC 6.x are not reflected in the DJCB and therefore are also not reflected in the PIA.
  • DFC PIA and DJCB were officially deprecated in the EMC Documentum 6.0 platform release (aka D6).
  • Since D6, EMC Documentum Foundation Services (DFS), is the fully supported and recommended platform API for interoperable solutions development (e.g. using .NET frameworks such as ASP.NET or Windows Forms).

Interoperability Review Details

Most folks that know EMC Documentum business logic software is predominantly implemented in Java (e.g. DFC/BOF, UCF, ACS/BOCS, etc.). However, this should not indicate a lack of support for other platforms such as .NET. So, let’s talk about .NET support, first by reviewing the past and finally by focusing on the present.

One small step toward going digital

I admit that I still enjoy the printed page–always will. However, while I won’t be trading in my novels for an e-book any time soon, I am trying to “go digital” where magazines and periodicals are concerned.

Resources like the digital libraries of ACM and IEEE are wonderful, and most magazines offer or plan to offer a digital version (e.g. PDF download).

Still, I receive my weekly allotment of trade rags in printed form. Still, I apply old habits. This week, though, I was introduced to life in the 21st century: copiers can scan documents and email you the results in PDF form.

My first 21-page submission yielded a 2 MB (black & white) PDF document. Papers are beginning to disappear off my desk (i.e. piles are moving onto my hard drive). This could be good…

Yeah, I know…living under a rock…  :-)