BlockChain Fundamentals Part 1


Let’s take an example where Person A is transferring $50 to Person B.
Person A sends a request to his  Bank for initiating the transfer, Bank verifies the request and if everything is ok then subtracts $50 Person A ‘s account and adds $50 to the account of Person B. Then bank updates its ledgers to reflect these changes.

Transaction in a Centralized Ledger

If you look at any of the transaction that we do these days are mostly handled by one or more servers managed by a single entity. This entity could be your bank, a social media or an online shop, and they all do a standard action, record your transactions in a centralized database.

Centralized Ledger

You might already be thinking that these entities do not really use a single server or ledger to perform these transactions, that is true even if they have multi-geographic fault tolerant sophisticated server farms to store their ledgers they all are managed and controlled by a single entity. In other words, the entities who records these data has the full control to do any action with their data.

Person A in the above case has to trust that the bank he transacts with will act as he expects, Though this system has been working for some time, there are few drawbacks for this type of transactions and ledgers.

  1. The entity who owns the ledger has the full control over it and can manipulate the ledgers at it is on will without its customer’s permission.
  2. The records in this ledger can be easily tampered with by someone who has access to it, means if someone makes some malicious changes to this ledger that can affect everybody who is relying on that ledger. (for example, someone can hack into the bank’s centralized ledger and modify transactions).
  3. Another disadvantage is the single point of failure. For example, if the bank decides to shut down their service users will not be able to perform transactions.
    In the extreme case where a natural disaster wipes out their all data centers, all the transactions from the ledgers will be lost.
  4. Centralized Ledgers stores data in Silo’s, means, for example, your bank has its ledger, an auditor has his ledger, tax authorities have its ledgers, and they are never synchronized.
  5. Though it is a centralized ledger from the general point of view, the organizations will have to spend too much of money to build redundancy and scaling to their ledgers. That makes this very expensive

What is BlockChain

In simple terms, Blockchain is a distributed ledger of transactions. All the transactions in the blockchain are encrypted and synchronized between the participants.

Blockchain-workings-explained

Key points about Blockchain

  1. It is a Distributed Ledger.
  2. Members of a blockchain network are called Nodes.
  3. Each node has the copy of the full ledger.
  4. Nodes use Peer to Peer Network for synchronization.
  5. All information on the Distributed ledger is Secured by Cryptography.
  6. By design Eliminates the need for a Centralized Authority to validate transactions by performing peer validations before any transactions
  7. The transactions are added to ledger based on Consensus from nodes.
  8. Each valid transaction in the Blockchain is added to a Block 
  9. Blockchain miners create Block
  10. Multiple Blocks make a Blockchain.
  11. All Blocks in Blockchain are Immutable.
  12. It ensures the Complete Audit trail of the whole transactions (Verifiability)

I know this is too much to chew on, I promise you will get all these concepts by the end of this article. So be with me and let’s move on…

Distributed Ledgers  (DLT)

According to Wikipedia “A distributed ledger is a consensus of replicated, shared, and synchronized digital data geographically spread across multiple sites, countries, or institutions. There is no central administrator or centralized data storage.”Distributed Ledger

In the Distributed ledger each participating member has a copy of the ledger, In simple terms during any transactions, it updates the ledger of the sender and receiver then broadcast the transaction, The transaction details are updated in the ledgers of all of its participants using peer to peer network.

Peer to peer (P2P) network

Peer to Peer (P2P) is a decentralized communication model where each participating nodes will have the same capabilities. Unlike the client-server model, any node in the peer to peer network can send requests to other nodes and respond to requests by other nodes. The best example for a peer to peer network is BitTorrent.

Peer to peer Network

Any peer can perform a transaction on the P2P network this means When one node can perform a similar transaction at the same time it will end up in conflicts. This is called double-spend problem. Blockchain uses a consensus system to resolve these kinds of conflicts.

We will discuss the double-spend problem and consensus systems in detail in a later post. Let ‘s focus on the basic concepts first.

Cryptography in Blockchain

Due to its nature, any data in a Blockchain is visible to all the members of the network, this makes this data vulnerable and hackable, however, Blockchain uses cryptography to make all the transactions extremely safe and secure.

In simple terms, Cryptography is used for obfuscating (encrypting and decrypting) data. Blockchain leverages two cryptographic concepts in its implementation.
They are the following

  • Hashing
  • Digital Signature.

What is Hashing

Hashing is a mechanism where any input is transformed to a fixed size output using a hashing algorithm. The input could be of any file type, for example, you can generate a hash of an image, text, music, movie or a binary file.

Hashing-Representation

Whatever may be the size of the file, hashing algorithm guarantees that the output is of a fixed size. (For example, if you create an SHA 256 hash of any file the hash will always be 256 bits.)

Key properties of Hashing.

Any hashing algorithm should adhere to the following principles.

Determinism
  • For a given value the algorithm should always produce the same hash value.
For example, the SHA 256 Hash of the word "Hello World"  will be always
A591A6D40BF420404A011733CFB7B190D62C65BF0BCDA32B57B277D9AD9F146E
Pre-image resistance
  • This means it should be computationally hard or impossible to decrypt the input from the output.
In the above example, we saw the hash value
A591A6D40BF420404A011733CFB7B190D62C65BF0BCDA32B57B277D9AD9F146E
represents "Hello World". 
It is impossible to decrypt the word Hello World from the above hash value.
Second pre-image resistance
  • This means the hash generated with input should not match with a hash value of a hash value of a different input
    For example function hash(“Hello World”) != function hash(“XXX”) where “xxx” is any input other than word “Hello World”
Collision Resistance.
  • This means it should be hard or impossible to find two different input (of any size or type) to have the same hash. Collision Resistance is very similar to Second pre-image resistance.

Hashing is commonly used to find the checksum of a file. For example, when you are downloading software from a server the software vendor provides the checksum or Hash of the software package. If the hash of the downloaded software is matched with the hash given by the provider of the file, we guarantee that the software was not tampered with.

Blockchain uses Hashes to represent the current state of the blockchain. Each block in the blockchain may have hundreds of transactions and verifying each transaction individually is going to be very expensive cumbersome. So Blockchain leverages Merkle root to verify the transactions.

Merkle tree of a Block

In a Merkle tree, each non-leaf node will have the hash of their child nodes. Look at the below diagram to understand the concept of Merkle tree.

MerkleRoot

Each block in the blockchain has the Merkle root of its transactions and the hash of its previous block. The hash of Merkle root can be used as a definitive mechanism to verify the integrity of the block as even the slightest changes to any of the records in this tree will alter the value of the original Merkle Root.

Block Structure in BlockChain

In other words, the entire state of a Blockchain system can be validated by the hash of its last block which is of 256 bits.

What is a Digital Signature

A classical example of Digital Signature is the website traffic using HTTPS protocol using SSL. SSL uses the digital signatures to ensure the authenticity of the server.

A User generates a digital signature by generating a Public and Private key Pair.

Generated Key

A Public key an a Private key has mathematical relations that tie each other. The private key should be kept as secret and should be used for signing messages digitally. A public key is intended to be distributed publically which should be used by the message recipient to validate the authenticity of the message.

Sending a Message with Digital Signature

Sender signs all his transactions with his generated private key. This will ensure that only the owner of the account with the private key can do the transaction.

Verifiying a message with Digital Signature

The Reciever or any nodes in the Blockchain verifies the transaction by checking the digital signature of the transaction using the public key.

Key Points to Remember about Hashing in Blockchain

  • Hashing is used for verifying the integrity of the transaction
  • The digital signature is used for verifying the identity of the performer of a transaction. 

For learning more about the Cryptography and Hashing in Blockchain, please visit the following links

https://blockgeeks.com/guides/cryptocurrencies-cryptography/

https://blockgeeks.com/guides/what-is-hashing/


What is a Block

We saw that Blockchain is a group of blocks in sequential order, Let’s take a quick peek at what a block is? In simple terms Block is a group of valid and verified transactions. Each block in the blockchain is immutable. Block miners will continuously process new transactions, and new blocks will be added to end of the chain. Each block will have the Hash of the previous block thus ensuring the integrity of the chain.

Block Structure in BlockChain

Every block in a blockchain will have the following information

  • Hash of the previous Block
  • Timestamp at which the block was created
  • List of the transactions that were part of that Block
  • Merkle tree of all the transactions in that block
  • Nonce – (A random String generated by miner)
  • Hash of header of the block which will be used as the Hash of the previous block in the next block

Let’s take a look at a real example of a Block; As you all know Bitcoin is based on Blockchain, To explain a Block I am referring to a block from bitcoin transaction.  I had taken a real Block from Bitcoin Blockchain network from  www.blockchain.info

You can see that this block has a header and a list of transactions. Transactions and its details can vary depending upon the blockchain implementation since this is a Bitcoin block you will see Bitcoin transactions.

Bitcoin Block Sample

Let’s go through each some of the key datasets in this block header

Field name Summary
Block Id Unique ID representing this block
Number of Transactions Total number of transactions recorded in that block
Height Total number of blocks preceding to this block on that blockchain, (in this case, there are 505234 blocks created before this block)
Timestamp Time at which this block was created
Relayed By Miner who mined this block
Transactions This shows the hash of each transaction with its details

Genesis Block

We learned that any Block will always have the hash of its previous block attached to it. So its implied that First block (Block 0) of a Blockchain will not have a previous block, and this is known as Genesis Block. Genesis block is almost hardcoded into the applications that use that blockchain.

Click here to see the Genesis Block of Bitcoin

Block time

Block time is the time taken to mine a block. Block time varies from implementation to implementation in the blockchain. To provide security and prevent forking each implementation defines its own block time. For Bitcoin, the block time is 10 mins whereas for Etherium its around 20 seconds.

What is in a transaction

Again I am taking a bitcoin transaction to explain a transaction record in Blockchain.  The below transaction shows the transfer of a bitcoin from one address to two recipients which was part of the Block that you had seen in the previous image.
The Tree diagram below shows the related transfers of those bitcoins.  This related or audit trail of the asset (Bitcoin in this example) ensures the legitimacy of this transaction.

Bitcoin Transaction

What is Block Mining

In simple words, Miners are the one who runs a specialized version of the Blockchain software which can add a Block to the Blockchain. The miners get rewarded each time when they add a block to the Blockchain. There may be multiple miners who will compete with each other to create a Block in the blockchain in a network.

Blockchain Network with Nodes and Miners

Miners keep transaction pool of unconfirmed transactions tries to wrap around them to create a block. They will have to solve a mathematical puzzle to add the block to the blockchain. These mathematical puzzles are to create a hash of the block of a  particular nature.

The constants in this puzzle are the following

  • Previous Block Hash
  • Time Stamp
  • Merkle root

The variable in this puzzle is the Nonce.  A nonce is a fixed size string that can include both numerical and characters.

Miners keep trying new nonce till they solve this puzzle and the first miner who can solve this wins and broadcast to the network and the block is added to the chain. Then the nodes add this block to their ledger.

Currently, Bitcoin and Etherium use an algorithm called proof-of-work to mine a block,

Below is the illustration of the proof-of-work algorithm. 

Proof of work - Bitcoin Implementation

There are multiple algorithms used by blockchain network for mining and consensus,  We will discuss those algorithms in detail in a different post.

Few Usages of Blockchain

The blockchain is an emerging technology and there are a ton of use cases that we can solve with this.  Here are the few use cases that can leverage the power of Blockchain.

  • Health care records sharing
    Privacy of Personal health records is a major concern right now, think about a system where your personal health records can be stored in a blockchain and shared with the doctors within seconds.
  • Insurance Claim Processing
    We know that Insurance industry is prone to lots of fraudulent claims and fragmented sources of data.  Chances of error (intentional or unintentional) are very high. With the Blockchain, we can have more transparent and error-free insurance claims.
  • Payments and Banking
    The Major issue in the banking and payment sectors are fraud and money laundering. With the transparency Blockchain provides we should be able to eliminate most of these.
  • Voting systems
    We saw earlier that Blockchain is tamper proof or tamper evident. Implementing a blockchain based voting system can create an unhackable voting system.
  • Smart Contracts
    Smart contracts allow the self-execution of contracts. (I will cover smart contracts with examples in a different post). Blockchain not only eliminates the need for third-party for smart contract enforcement but also enforces the terms of a contract when the terms are met.

If you look at these use cases the blockchain will be a great choice if we auto validate the transactions using smart devices. In other sense, in order to leverage the full potential of Blockchain, we need to have more IoT sensors which can validate many of these “transactions”.

Food for thought

Imagine you go to a grocery store and pick up a bottle of organic, you are not sure whether this milk is really organic or not, you take your smartphone and scan the QR code of the batch id of the milk, The application lists out the details of dairy farm along with the cattle food they are buying, health history of the all the cows in the farm along with their medical records where you can trace back to each and every detail of what the farm states about the milk… that is the future that Blockchain  can offer.

Implementations

If you look at the Gartner Hype Cycle for emerging technologies 2017 you can see that Blockchain is slowly moving to Trough of Disillusionment and they are expecting this to mature to mainstream adaptation in 5 to 10 years. My feeling is this could be shorter.

Emerging Technology Hype Cycle for 2017_Infographic_R6A
Picture Courtesy – Gartner.com

Having said that, there are a ton of development going on in the Blockchain field, Both large and small scale players are bringing lots of innovation to the blockchain technology.

Below are few of  the current major implementations of blockchain

Bitcoin

I don’t think Bitcoin needs an introduction, It is the first digital cryptocurrency and leverages the blockchain technology.

Ethereum

Etherium is an open-source blockchain platform for Blockchain applications and smart contracts. It is the first majorly accepted Blockchain platform. We will cover more on Ethereum in a different post.

We covered the basics of Blockchain here, This is just a beginning. Stay tuned more articles where we will get to more depth on some of the topics that we discussed here.

Thanks for reading and Please leave your feedback in the comments section.

Exploring MongoDB Stitch… Backend as Service !!!


In my long software development career, I always felt that the most significant time taking task is not building the actual business logic, but the amount of code that you need for your basic housekeeping tasks or in other words the “basic chores” of an application developer.

Initially, these chores included session management, memory management, thread management, etc. With the introduction of boilerplate codes and frameworks, etc. these chores have been drastically reduced. Thus making and the development teams to focus more on the core business functionality.

However, most of the team end up re-inventing the wheel by writing these lots of essential features over and over again such as user Authentication, sending notifications to your customers, etc.

If the above mentioned is the story of Enterprises, the story is a lot worse for startups, The Problem of unwanted chores is a big predicament for them, as most of the time, they are starting the scratches.

If you look at the Basic chores, they include the following.

  • Authentication
  • CRUD of Data
  • Fine-Grained data access
  • Integrations with other services

Now looking at the overall cost of these chores it is not just the development time and effort, but the increased the code complexity, Testing efforts, etc.

MongoDB Stitch

MongoStitch is meant to address precisely these problems. In the last Annual Developer conference, MongoDB introduced MongoDB stitch as an addition Mongo Atlas, their Cloud-based “database-as-service.”

Though Atlas is available on most of the platforms, MongoStitch is currently supported only on AWS US East 1 region and its tailored as an add-on to the existing  MongoAtlast subscription.

Mongostitch allow you to create an application from your Atlas console, configure it by enabling the following

  • Add new features to your existing application
  • Control the access to data for user
  • Integrate with other services

Once you set up your MongoStitch application on the Console, you can use create its client application and start calling the MongoStitch functionalities from your application using the stitch client.

Currently, Stich clients are available for the following platforms

  • Browser and Node (JavaScript)
  • Android Application (java)
  • iOS Application (Swift)

Mongo had done an excellent job of providing detailed documentation, and you can use the getting started guide to build  sample applications

Though MongoDB Stitch is still in Beta, the features it offers looks very promising. Let us explore few of these features.

Collection/ Field-level permissions

MongoDBStitch allows the developer to specify the access for the collections.  These access rule could be defined either for the collection itself or can be specific to each field in the collection, Though you can specify these rules be aware that they will be overridden by the access that you would have provided at the MongoDB level

Stitch Admin Console 2018-01-04 19-59-16

Service Integrations

This is my favorite part, Integrating with other services are a breeze. As of Jan 2018, the following service integrations are supported by Stitch

Service Name Supported Services
S3 Upload file to S3, generate signed URL
Amazon SES Send Email
Github Webhooks
HTTP Services Basic HTTP calls (get, post, delete, put, patch, head)
GCM Push notifications to Apple and Android devices
Twilio Send and Receive Text Messages

Each of these service calls can be configured to have its own rules

Stitch Admin Console Service Integration

Authentication Services

User Authentication services using the following

  • E-mail and inline password
  • Google
  • Facebook
  • API Keys
  • Custom/Third party Authentication.

These integrations are super easy, and I was able to create a google and facebook login integration for my sample app within 15 minutes.

Stitch Admin Console Authentication

Values (Constants)

These are named constants that you can use in Stitch functions and rules.

Stitch Admin Console Values

Functions

The Functions in MongoDB stitch is written in Javascript and can be edited and tested using built-in function editors.

As of now, the ECMAS version 6 is not supported in functions.

Stitch Admin Console 2018-01-06 12-32-24


After playing around with MongoStich for a few days, I feel that this has lots of potentials and it can definitely improve productivity and helps you to focus on the core business logic.

Final Words

During my POC I wanted to extend my logged in users with additional attributes say for example I wanted to capture the address and phone numbers of my user,  however, MongoStich is saving the users in a different mechanism that is not really extendible and not visible as a collection.

Stitch Admin Console users

If Mongostitch allows the user information to be saved to a table that can be extended with any extended attributes that I wanted to add to that users it will make the life more easier for developers.

Here are some useful links on MongoDBStitch

Documentation: https://docs.mongodb.com/stitch/

Tutorials and Getting Started Guide: https://docs.mongodb.com/stitch/getting-started/

Let me know your thoughts …

Setting up Cloudwatch for Custom logs in AWS Elastic Beanstalk


Amazon Cloudwatch monitoring services are very handy to gain insight into your application metrics,  besides metrics and alarms you can use this to go through your application logs without logging into your server and tail the logs.

I ran into few issues when I was initially setting up Cloudwatch for my custom logs in the Elastic Beanstalk  Tomcat Application.  I will walk you through the whole process on this blog.

Setting up your application

In this example, I am using a Spring boot Application which will be deployed in ElasticBeanstalk Tomcat container.

.ebextension file

First, you need to create a .ebextention file for your application
Here is a working sample of the .ebextension file


files:
"/etc/awslogs/config/mycustom.conf" :
mode: "060606"
owner: root
group: root
content: |
[/var/log/tomcat8/mycustomlog.log]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat8/mycustomlog.log"]]}`
log_stream_name = {instance_id}
file =/var/log/tomcat8/mycustomlog.log*

The above configuration will create a custom configuration to copy logs from /var/log/tomcat8/mycustomlog.log to a log group named for my application and will copy over all the logs with the pattern mycustomlog.log

This line creates a configuration file mycustom.conf in the /etc/awslogs/config/mycustom.conf location. Once deployed you can SSH to this location to view your configuration.


files:
"/etc/awslogs/config/mycustom.conf" :

The following lines create the log groups and create the scripts to copy over the files to cloudwatch


content: |
[/var/log/tomcat8/mycustomlog.log]
log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/tomcat8/mycustomlog.log"]]}`
log_stream_name = {instance_id}
file =/var/log/tomcat8/mycustomlog.log*

Make sure that you check your .ebextension is a valid yaml before deploying this to your application environment.  I use http://www.yamllint.com/ to check the validity of my YAML’s 

Place your .ebextension file in the /src/main/resources/ebextensions/ folder of your project

Screenshot1

Gradle Script

Now you need to update your Gradle scripts to make sure that you package your .ebextnsion file along with your war file

Update your Gradle Script to include the ebextension in the root of the file


war {
       from('src/main/resources/ebextensions') {
       into " .ebextensions";
   }
}

With this gradle script, your war file should have a .ebextensions folder in the root and should have the mycustom.conf file in it.

Now let’s prepare your Elastic Beanstalk to enable the cloudwatch

Prepping up your Elastic Beanstalk  Environment

To enable Cloudwatch for Elastic Beanstalk you need the following

  1. Permission for Elastic Beanstalk to create log group and log stream
  2. Enable the Cloudwatch on the Elastic Beanstalk application

Login to your AWS Account, go to IAM and create a new Policy  similar to the following

Grant Permission to Elastic Beanstalk

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "CloudWatchLogsAccess",
            "Action": [
                "logs:CreateExportTask",
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:DescribeDestinations",
                "logs:DescribeExportTasks",
                "logs:DescribeLogGroups",
                "logs:FilterLogEvents",
                "logs:PutDestination",
                "logs:PutDestinationPolicy",
                "logs:PutLogEvents",
                "logs:PutMetricFilter"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:logs:*:*:log-group:*"
            ]
        }
    ]
}

Now attach this policy to “aws-elasticbeanstalk-ec2-role”

Enable CloudStream on your Elastic Beanstalk application

Go to your Elastic Beanstalk Application, Edit Software Configuration in the Configuration Menu

Configuration 2017-12-10 16-21-55

Enable Cloudwatch Logs from the settings

Configuration 1 2017-12-10 16-21-55

Once you do this the AWS will re-configure the system, now you deploy the war file created from the Gradle script.

Usually, AWS picks up the configuration after you deploy the new war file. if not restart the environment.

Go to the cloudwatch to verify your log stream

Troubleshooting Tips

As I said before I had issues while I was setting this up, if your configurations are not getting picked up go ahead with the following steps to  troubleshoot this issue

  • Make sure that your YAML is valid.
  • SSH into the Environment and make sure that the file created in the location /etc/awslogs/config/mycustom.conf is valid.
  • Check eb-publish-logs.log to see if it has any errors
  • Finally, if nothing works rebuild your environment.

Service Based Objects (SBO’s) in Documentum


Documentum Business Object Framework which was introduced from Documentum 5.3 plays a key role in most of the current Documentum implementations.  Service-based Object is one of the basic members of Documentum BOF family.  Let’s try to see what makes Service Based Objects very popular and how can you implement it.

What is an SBO

In simple terms, SBO in Documentum can be compared to session beans of the J2EE environment.  SBO enable the developers to concentrate just on the business logic, and all the other aspects will be managed for you by the server. This reduces the application code significantly and reduces lots of complexities. The most significant advantage of a BOF that it’s deployed in a central repository. The repository maintains this module and DFC ensures that the latest version of the code is delivered to the client automatically.

Service-Based Objects are repository and object type in-depended that means the Same SBO can be used by multiple Documentum repositories and can It can retrieve and do operations on different object types. SBO’s can also access external resources, for example, a Mail server or an LDAP server. Before the introduction of Documentum Foundation Services, SBO’s were commonly used exposed to expose Documentum web services.

An SBO can call another SBO or by any Type based Objects. (Type Based Objects (TBO) are a different kind of Business Object types which I will explain in a separate study note)

A very simple to understand example for an SBO implementation would be a Zip code Validator. Multiple object types might have Zip code across multiple repositories.  So if this functionality is exposed as an SBO, it can be used by the custom application irrespective of Object types and repositories. This Validator SBO can be used even by different TBO’s for validations.

Here are some bullet points about SBO’s for easy remembering

  • SBO’s are part of Documentum Business Object framework
  • SBO’s are not associated with any repositories
  • SBO’s are not associated with any Documentum object types.
  • SBO information is stored in repositories designated as Global Registry.
  • SBO’s are stored in /System/Modules/SBO/<sbo_name> folder of repository. <sbo_name> is the name of SBO.
  • Each folder in /System/Modules/SBO/ corresponds to an individual SBO

How to implement an SBO using Composer

The steps to create an SBO are these.

1) Create an interface that extends IDfService define your business method
2) Create the implementation class implement write your business logic, This class should extend DfService and implement the interface defined in Step 1
3) Create a jar file for the created Interface and another jar for the implementation class then create Jar Definitions
4) Create an SBO Module and Deploy your Documentum Archive using Documentum Composer (Application builder for older versions)

Let’s see these steps with an Example SBO Zip Code Setter, I am not covering the steps using application builder here. The screenshots and the notes will give you an insight into how to use Documentum Composer to implement a Service Based Object in Documentum version 6 or above.

Step 1: Create an interface and define your Business method

The first step is to create an interface which will define the business functionality. This interface should extend IDfService interface. The client application will use this interface to instantiate the SBO.

Click New –> Interface in Documentum Composer. Click on the Add button of Extended Interfaces and search for IDfService. Select IDfService and click OK

image

Now Add the Business method ValidateZipCode() to an interface. The code should look like the following.

package com.ajithp.studynotes.sbo;

import com.documentum.fc.client.IDfService;
import com.documentum.fc.client.IDfSysObject;
import com.documentum.fc.common.DfException;

public interface IZipValidatorSBO extends IDfService {

public void validateZipCode (IDfSysObject obj, String zipCode, String repository)throws DfException;
}
Step 2: Create the implementation class

All the Service Based Object implementation classes should extend from DfService class and implement the Interface created in the first step.  DfService class is an abstract class There are few methods which were abstract in 5.3 and has provided with a default implementation in 6.0 and later

Method Name Returns More information
getVendorString() String This method’s default implementation returns a empty String. Override to make changes to it.
getVersion() String This method returns a version which is not right, Override this method to return Major.minor version.
isCompatible() boolean The default implementation returns true if the version is an exact match

Let’s see some other important methods of DfService Class before we move further.

Method Name Returns More information
getName() String This returns the fully qualified logical name of the service interface
getSession() IDfSession This method returns IDfsession Object for the docbase name which is passed as argument to this method. You have to make sure that you call releaseSession() after you are done with the operation that involves session.
releaseSession() Releases the handle to the session reference passed to this method.
getSessionManager() IDfSessionManager Returns the session manager.

Managing repository sessions in SBO As We saw the previous table its always good practice to release the repository session as soon as you are done with its use. So the ideal way to do this should be like this.

// Get the session 
IDfSession session = getSession(repoNam);
try {
// do the operation with session
} catch (Exception e){
// Process the exception 
}finally {
// release the session 
releaseSession(session)
}

Transactions in SBO

Another important thing is to know is how to handle transactions in SBO. Note that only session manager transactions can be used in an SBO. The system will throw an Exception when a session based transaction used within an SBO.

beginTransaction() will start a new Transaction and use commitTransaction() to commit it or abortTransaction() to abort a transaction.  Always ensure that you are not beginning a transaction where another transaction is active. You can use isTransactionActive() to find out whether a transaction is active or not.

Another important point is if your SBO doesn’t start a transaction don’t commit it or abort it in the SBO Code instead if you want to abort the transaction use setTransactionRollbackOnly() method.

Other important points

1) Since SBO’s are repository independent, do not hardcode the repository names in the methods. Either pass the repository name as a method parameter or have it as a variable in SBO and use a setter method to populate it after instantiating

2) Always try to make SBO’s stateless (Its a pain to manage state full SBO’s ).

3) Don’t reuse SBO, Always create a new instance before an operation.

Now let’s see how to code our ZipSetterSBO

Click on New –> Class, Click on the Browse button of Superclass and Search and Select DfService and in the Interfaces search for the Interface created in the previous step and Click OK. Also, select the option Inherited Abstract Methods in Which method stubs would you like to create.

image

I had overridden method getVersion() for the illustration purpose. See the code sample for the inline comments.

package com.ajithp.studynotes.sbo.impl;

import com.ajithp.studynotes.sbo.IZipValidatorSBO;
import com.documentum.fc.client.DfService;
import com.documentum.fc.client.IDfSession;
import com.documentum.fc.client.IDfSysObject;
import com.documentum.fc.common.DfException;

public class ZipValidator extends DfService implements IZipValidatorSBO {

public static final String versionString = "1.0";
// overriding the default 
public String getVersion() {
        return versionString ;
      }

public void validateZipCode (IDfSysObject obj, String zipCode, String repository) throws DfException {
     IDfSession session = getSession(repository);
     try {
     if (isValidUSZipcode(zipCode)){
         obj.setString("zipcode",zipCode);
         obj.save();
      }
     } catch (Exception e){
         /* Assuming that transaction is handled outside the code and this says DFC to abort the transaction 
         in case of any error */
        getSessionManager().setTransactionRollbackOnly();
        throw new DfException();
     } finally {
     releaseSession(session);
    }
  }
 private boolean isValidUSZipcode(String zipCode){
     // implement your logic to validate zipcode. 
     // or even call a external webservice to do that 
     // returning true for all zip codes
      return true;
   }
}
Step 3: Generate Jar files and Create Jar Definitions

The next step in SBO creation is to create Jar files which will hold the interface and the implementation classes. These jar files are required to deploy your SBO.

Use Composers/Eclipse Create Jar option or command line jar command to create the jar file

image image

image

Selecting the sbo package to create the interface jar

image

Selecting the com.ajithp.studynotes.sbo.impl for implementation.

Look at the Composers Export Jar screenshots for Interface and implementation (Refer Eclipse Documentation for more details). I think the figures posted above are self-explanatory.

The Command line to create a Jar file is jar cf <name_of_jar>, Please look at the Java Documentation for more details on switches and options of Jar command.

The creation of Jar Definitions is new step added in Composer.

1) In Composer change the perspective to Documentum Artifacts Click New –> Other –> Documentum Artifacts –> Jar Definition

image

2) Click Next  and Enter the name of for the Jar Definition and click Finishimage

3) Select Type as Interface if the jar has only interface, Implementation if the jar has the only implementation of interface or Interface and Implementation if the single jar file has both interface and implementation. Click on the Browse button and browse to the jar created in the last step.

In Our case create two Jar Definitions The first one with type as Interface pointing to Jar Created for SBO and a second one with type Implementation pointing to the implementation jar

untitled

Name the Interface jar def as zipcodevalidator and the implementation jardef as zipcodevalidatorimpl

Step 4: Create a Module and Deploy the SBO

In Composer change the perspective to Documentum Artifacts then Click New –> Other –> Documentum Artifacts –> Module

image

Give a valid name and leave the default folder and Click Finishimage

In the Module, edit window select SBO from the dropdown

image

Now Click on Add Section of Implementation Jars of Core Jars. A new pop up window will appear which will have a list of all the Jar definitions set to Type Implementation and Interface and Implementation. Select the one you wanted to use for ZipCodeValidatorSBO that is ZipCodeValidatorImpl.

image

Click on the Select Button near pointing to Class name and Select the implementation class. In this case ZipValidator

image

Now Click on Add Section of Interface Jars of Core Jars. A new pop up window will appear which will have a list of all the Jar definitions set to Type Interfaces and Interface and Implementation. Select the one you wanted to use for ZipCodeValidatorSBO that is ZipCodeValidator.

image

For more details of other options refer to Documentum Composer Manual. Save the Module.

Now right click on the project and install the Documentum project

image

Click on the Login button after logged in Click on Finish to start the installation.

image

 

Look at the Documentum composer documentation to know more about the Installation options.

How to use SBO from a Client Application

follow the below steps to instantiate an SBO from a client application.

1) Get the Local client

2) Create login info and populate the login credentials.

3) Create an IDfSessionManager object

4) Use the newService () from the Client Object to create an SBO instance

// create client
  IDfClient myClient = DfClient.getLocalClient();
  // create login info
  IDfLoginInfo myLoginInfo = new DfLoginInfo();
  myLoginInfo.setUser("user");
  myLoginInfo.setPassword("pwd");
  // create session manager
  IDfSessionManager mySessionManager = myClient.newSessionManager();
  mySessionManager.setIdentity("repositoryName", myLoginInfo);
  // instantiate the SBO
  IZipValidatorSBO zipValidator = (IZipValidatorSBO) myClient.newService( IZipValidatorSBO.class.getName(), mySessionManager);
  // call the SBO service
  zipValidator.validateZipCode(obj, zipCode, "repositoryName");

Download this Study Note (PDF)

Using Java reflection to reduce Code and Development time in DFS


 

Java reflections are one of the most powerful API’s of Java Language, this can be used to reduce code significantly.

Most of the Current Enterprise application consists of different layers and they use Value objects to transfer data from one layer to another. An inefficient way of using getters and setters of the attributes of Value objects can increase code and development time of application. Effective use of reflection can reduce code and development time significantly.

So let’s take a Scenario,  I have an Object type MyObjectType extending from dm_document with 50 additional attributes, so dm_document as of Documentum 6.5 has 86 attributes adding additional 50 attributes that means we have 139 attributes for this object type. Consider a standard Web Application using DFS behind which needs to manipulate (add or edit) instances of this object type, The Service needs to add all these attributes to the PropertySet  of the DataObject representing that instance. Then need to call the appropriate service.

 

Considering that the bean instance name of MyObjectType is myObjectBean the Standard code will  be something like this

  ObjectIdentity objIdentity = new ObjectIdentity("myRepository");
  DataObject dataObject = new DataObject(objIdentity, "dm_document");
  PropertySet properties = dataObject.getProperties();
  properties.set("object_name", myObjectBean.getObject_Name());
  properties.set("title", myObjectBean.getTitle()); 
  // omited for simplicity


  objectService.create(new DataPackage(dataObject), operationOptions);

 

In the above code you have to explicitly set individual attributes for the object, the more the number of attributes the more complex and messy code.

Take another Example, where you have to retrieve an Object information and pass it over to the UI layer.

 myObjectBean.setObject_name(properties.get("object_name").getValueAsString());
 myObjectBean.setTitle(properties.get("title").getValueAsString());
 myObjectBean.setMy_Custom_Property(properties.get("my_custom_property").getValueAsString());

This operation can be more complex if you decide to use match the Data Type of your bean with the Object type.

 

So what is the best approach to reducing this complexity? the answer is the effective use of reflection API.

Let’s take a step to step approach to handle this issue.

To understand this better consider the below as the attributes of mycustomobjecttype

 

Attribute Name Attribute Type
first_name String
last_name String
age integer
date_purchased time
amount_due double
local_buyer boolean

 

Java Bean

Create a Java Bean that matches the Object Type

 public class Mycustomobjecttype {
  protected String first_name ;
  protected String last_name  ;
  protected int age;
  protected Date date_purchased  ;
  protected double amount_due  ;
  protected boolean local_buyer ;
  public int getAge() {
    return age;
  }
  public void setAge(int age) {
    this.age = age;
  }
  public double getAmount_due() {
    return amount_due;
  }
  public void setAmount_due(double amount_due) {
    this.amount_due = amount_due;
  }
  public Date getDate_purchased() {
    return date_purchased;
  }
  public void setDate_purchased(Date date_purchased) {
    this.date_purchased = date_purchased;
  }
  public String getFirst_name() {
    return first_name;
  }
  public void setFirst_name(String first_name) {
    this.first_name = first_name;
  }
  public String getLast_name() {
    return last_name;
  }
  public void setLast_name(String last_name) {
    this.last_name = last_name;
  }
  public boolean isLocal_buyer() {
    return local_buyer;
  }
  public void setLocal_buyer(boolean local_buyer) {
    this.local_buyer = local_buyer;
  }
}

Getting the Values from PropertySet (Loading Java Bean)

……

List<DataObject> dataObjectList = dataPackage.getDataObjects();
DataObject dObject = dataObjectList.get(0);
Mycustomobjecttype myCustomObject = new Mycustomobjecttype();
populateBeanFromPropertySet(dObject.getProperties(),myCustomObject);

……

// See the Reflection in Action here 
public void populateBeanFromPropertySet(PropertySet propertySet, Object bean)
  throws Exception {
 BeanInfo beaninformation;
 beaninformation = Introspector.getBeanInfo(bean.getClass());
 PropertyDescriptor[] sourceDescriptors = beaninformation.getPropertyDescriptors();
 for (PropertyDescriptor descriptor : sourceDescriptors) {
     Object result = null;
     String name = descriptor.getName();
    if (!name.equals("class")) {
      if (propertySet.get(name) != null) {
        if (descriptor.getPropertyType().getName().equals("int")) {
          result = new Integer(propertySet.get(name)
              .getValueAsString());
        } else if (descriptor.getPropertyType().getName().equals("double")) {
          result = new Double(propertySet.get(name).getValueAsString());
         } else if (descriptor.getPropertyType().getName().equals("boolean")) {
          result = new Boolean(propertySet.get(name).getValueAsString());
         } else if (descriptor.getPropertyType().getName().equals("java.util.Date")) {
          DateProperty dat = (DateProperty)propertySet.get(name);
          result = dat.getValue();
        }else {
          // none of the other possible types, so assume it as String
          result = propertySet.get(name).getValueAsString();
        }
        if (result != null)
          descriptor.getWriteMethod().invoke(bean, result);
      }
     }
  }
}

Setting Values to Property Set

 

public DataPackage createContentLessObject(Mycustomobjecttype myCustomType) throws Exception {
ObjectIdentity objectIdentity = new ObjectIdentity("testRepositoryName");
DataObject dataObject = new DataObject(objectIdentity, myCustomType.getClass().getName());
PropertySet properties = populateProperties(myCustomType);
properties.set("object_name",myCustomType.getFirst_name()+myCustomType.getLast_name() );
dataObject.setProperties(properties);
DataPackage dataPackage = new DataPackage(dataObject);
OperationOptions operationOptions = new OperationOptions();
return objectService.create(dataPackage, operationOptions);
}

 

// Reflection in Action  
public PropertySet populateProperties(Object bean)throws Exception {
BeanInfo beaninfo;
PropertySet myPropertyset = new PropertySet();
beaninfo = Introspector.getBeanInfo(bean.getClass());  
PropertyDescriptor[] sourceDescriptors = beaninfo
      .getPropertyDescriptors();
  for (PropertyDescriptor descriptor : sourceDescriptors) {
    String propertyName = descriptor.getName();
    if (!propertyName.equals("class")) {
        // dont set read only attributes if any
       // example r_object_id 
       if (!propertyName.startsWith("r")) {
        Object value = descriptor.getReadMethod().invoke(bean);
       if (value != null) {
          myPropertyset.set(propertyName, value);
        }
      }
   }
 }
  return myPropertyset;
}

Chaining of Custom Services in DFS


 

There is an interesting drawback in Documentum Foundation Services Version 6.5,

Issue:

When you chain custom services and try to build the Services the build fails lets see a Scenario from the DFS sample code itself

@DfsPojoService(targetNamespace = http://common.samples.services.emc.com&#8221;, requiresAuthentication = true

) public class HelloWorldService

{

public String sayHello(String name)

{

ServiceFactory serviceFactory = ServiceFactory.getInstance();

IServiceContext context = ContextFactory.getInstance().getContext();

try {

IAcmeCustomService secondService = serviceFactory.getService(IAcmeCustomService.class, context);

secondService.testExceptionHandling();

} catch (ServiceInvocationException e) {

e.printStackTrace();

} catch (CustomException e) {

e.printStackTrace();

} catch (ServiceException e) {

e.printStackTrace();

}

return “Hello “ + name;

}

}

Here in the sample code of DFS I am chaining the services, Here everything looks fine and when you now you build this service during the genarateArtifacts ant task the Build will fail with a will get a ClassNotFound compiler error at

IAcmeCustomService secondService = serviceFactory.getService(IAcmeCustomService.class, context);

What happens here is when the build does the initial clean up all the generated Client interfaces are deleted and DFS currently not checking for any dependencies.

Let me take the example of dfs-build.xml that’s the part of CoreDocumentumProject in composer

<generateArtifacts serviceModel=“${gen.src.dir}/${context.root}-${module.name}-service-model.xml” destdir=“${gen.src.dir}/”>

<src location=“${src.dir}” />

<classpath>

<path refid=“projectclasspath.path” />

</classpath>

</generateArtifacts>

</target>

 

In this we cannot set any exclusion path in <src location=“${src.dir}” />

Simply because it even if you provide <fileset/> or <direst/> with pattern set its not recognizing it.

I had raised a support case with EMC and they told me that this is not currently supported!!!! And they will add this as a feature request

This means we cannot Chain Custom Services unless EMC fix this or we do a semi manual workaround to overcome this issue.

The Work-around that I found

Follow these steps to overcome this issue

Step 1,

Identify the Services those will call the custom services, and create a new source directory for it in composer, here I am calling them as depended_src and move the services that calls the custom services to there, the depended src should be in a separate path than the webservices- src

src-img1

Step 2

1) Now Edit the Build file and add these two properties

 

<property name=“my.core.services.classes” value=“${service.projectdir}/Web Services/bin/classes” />

 

<property name=“dep.src.dir” value=“${service.projectdir}/depended_src” />

The dep.src.dir should point to the depended src location mentioned in step 1

2) Create an additional target for generatemodel and generate artifacts

<target name=“generateDependencies” depends=“generate”>

<echo message=“Calling generateDependencies” />

<generateModel contextRoot=“${context.root}” moduleName=“${module.name}” destdir=“${gen.src.dir}/”>

<services>

<fileset dir=“${dep.src.dir}”>

<include name=“**/*.java” />

</fileset>

</services>

<classpath>

<pathelement location=“${my.core.services.classes}” />

<path refid=“projectclasspath.path” />

</classpath>

</generateModel>

<generateArtifacts serviceModel=“${gen.src.dir}/${context.root}-${module.name}-service-model.xml” destdir=“${gen.src.dir}/”>

<src location=“${dep.src.dir}” />

<classpath>

<pathelement location=“${my.core.services.classes}”/>

<path refid=“projectclasspath.path” />

</classpath>

</generateArtifacts>

<!– signal build is done –>

<!– used by DFSBuilder.java –>

<copy todir=“${src.dir}/../” file=“${basedir}/dfs-builddone.flag” />

</target>

3) Now edit dfs-build.properteis and add the following property

service.projectdir= <absolute path to the project>

Step 3

1) Run the generate task,

2) Copy all the service entries from (between <module> and </module><context-root>-<module-name>-service-model.xml you can find this in <project_dir>\Web Services\bin\gen-src folder

3) Now run the generateDependencies task that was created on Step 2

4) Now Edit <context-root>-<module-name>-service-model.xml and add the copied services to this file

5) If you want to create the jar files now you can call the package task after this.

This should help you to chain custom services , and if you found any alternate ways please comment.

 

Federation in Documentum


Federation is one among the most common distributed Documentum model. This means multiple Documentum repositories run as a federation. There will be a Governing repository and multiple member repositories in this model. Lets try to find out more about Federation

 

Take this typical scenario A Major Pharmaceutical Company ABC Corporation has multiple research centers and production plants across the glob and they have multiple Documentum repositories used for storing various information. A user logged into a corporate application needs to fetch documents from these various repositories in a Single session. Each repository in this scenario should have same set of users, groups and ACL for this architecture to work, manually managing these kind of scenario is trouble some and error prone.

 

Now lets see what a federation can do to make it less complex.

As I mentioned above Federations consists of Governing and Member repositories all the changes that has been made to global users and groups and external ACLS in the governing repository are automatically reproduced in the member Repository.

 

Requirements for Federation

·         Object types definition should be same in the all participating repositories.

·         User and group definition should be same in all participating repositories.

·         The server on which governing repository runs must project to the connection brokers at the servers where member repository runs

·         The server on which member repositories runs must project to the connection brokers at the servers where governing repository runs

·         If any of the participating Content Servers are with trusted server licenses Either
The servers should be configured to listen on both secure and native port or
The secure connection default for clients allows the clients to request a connection on a native or secure port

 

Few Bullet points about Federation

·         Any alteration done to any of the object type will not be automatically pushed to the participating repositories

·         Only users or groups marked as Global while creating them will be pushed / synchronized with participating repositories

·         The users those are part of any object types that are extended from dm_user will not automatically pushed. This will happen only if you specify this type in the Federation configuration.

·         Each repositories can be part of a single federation

·         A federation may contain different Content Server versions

·         A federation may contain a mix of trusted and non-trusted Content Servers.

 

 Download this Study Note (PDF)