Darwino as the IBM Domino Reporting Holy Grail

Reports, dashboards, and data analytics have been the bane of IBM Notes/Domino since the beginning. Its proprietary data structure, the absence of standard APIs, and its deficient query capability make reporting very difficult. This has been ranked as one of the top needs for any for business applications. I know several business partners who created great Domino solutions but struggle with poor reporting capabilities.

Of course some attempts were made to fix it: LEI, DB2NSF, etc., but they are all incomplete and defunct now. I personally tried a new approach with DomSQL, which, to be frank, I was proud of. But this was a first shot, more a proof-of-concept than a product, although I know several customers using it in production. Unfortunately, despite the community interest, IBM didn’t want to move it forward, so it is barely in the state where I left it. On the positive side, it gave me a great low level SQLite experience which is now leveraged in Darwino’s mobile database! Learn, learn, learn: you’ll always reuse it.

So we’re back to where we were twenty years ago, with no solution to this common business problem. As you might have anticipated, Darwino can help! It is actually the first, most immediate benefit that our customers are getting when they start with Darwino.

One of Darwino’s capabilities is data replication. It can replicate, two ways and in real time, any Domino data into its own document store. From a Domino standpoint, Darwino is behaving as another Domino server, with high-fidelity replication. The Darwino document store is a JSON based store, sitting on top of an RDBMS. As modern RDBMS now understand JSON natively, this is a great fit. The whole replicated data set is then query-able by any tool that understands RDBMS, with no limitation but the relational database itself. Yes, you can join, aggregate, union, calculate, select, sub select, anything. And, contrary to LEI, this is a real incremental replication that can be triggered when a data event occurs on the Domino server, keeping your data up-to-date at any time. No more nightly/weekly batch data copy, if you know what I mean.

From a technical standpoint, Darwino stores a JSON version of the Domino document within a column in a relational table. It is very flexible as it does not require a custom table schema per fdata set, and there is no data loss (e.g. a field you forgot to define a column for). Moreover, if the default field conversion between the two database is 1-1, this is actually fully customizable. The Darwino replication process can transform the data during the replication phase. This is achieved using a Groovy-based DSL, which allows the following:

• The fields type can be normalized (e.g. make sure that the field in the JSON document is of the expected data type). Common issue with Notes inconsistent data type across documents, right?
• Domino patterns/workaround can also be normalized. For example, a set a multi value fields describing a table can be transform a true JSON array
• Computation can be done ahead of time to make the reporting easier (queries, calculated columns…)
• Well, your imagination!

If you’re uneasy about directly reporting on the JSON column within the RDBMS , then you can create RDBMS views (standard or materialized), to then act on a table-like structure. This is what our partner, CMS, chose to do. Going even further, they are creating these relational views out of the Domino view design elements. Introduction to their product is available here. This is a great use of Darwino.

In summary, using Darwino as a Domino report tool enabler is a fast way to fulfill some crucial, missing Domino capability. It almost instantly makes the existing Domino applications more interesting to the business users. Of course, this is just a subset of what Darwino can bring to the table, but it’s a quick, high-value one. Please, give it a shot or contact us so we can help you.