I spent the first decade of my career at Microsoft. As a result, the only stack I was familiar with was Microsoft SQL Server at the backend, an API layer using SOAP + XML in the middle, topped with a web layer built on .Net.
I was drunk on the SOAP kool-aid and completely ignored the inefficiencies created by SOAP + .NET. For example, the view state transferred for every interaction between the API and Web layer was very heavy and led to the following:
Complicated stored procedures: We tried to minimize calls between Web and API layers, which meant that any call that retrieved complex information from multiple tables needed a SQL Server stored procedure.
Multiple APIs to manage CRUD: API contracts did not represent the DB schema and multiple CRUD APIs interacted with the same database object. This led to confusion among developers and frequent regression issues since it was difficult to find all code locations where an object was being created or updated.
Fragile database: The above issues made us reluctant to change anything in the database since it caused bugs and regressions. This meant our database was virtually frozen.
Having experienced this as a developer, manager, and eventually a Product Unit Manager, the first thing I did at Shippable was to pledge total allegiance to REST. One of the most important principles of REST is that every object should have an http routable method. Now this led us to a very interesting conundrum: where should we compose the objects that need to be displayed in the UI? Should we build a layer of finished APIs that return a ready-to-display object or should we compose the object in the UI layer by making multiple calls to the basic CRUD APIs?
The former approach of building a layer of finished APIs has the following advantages:
- the web layer does not need to make multiple calls to API to get all the data it requires
- the UI team doesn't need to understand how to compose the required objects
However, the disadvantages are similar to using SOAP with stored procedures, except that the view state is no longer a problem. Then I started to examine the advantages: Is it really that bad to make multiple calls from UI to API and compose the objects in the UI layer?
Based on my findings, this universal assumption isn't true for most scenarios. Let's look at the problem mathematically and see why.
Assume that there are 4 objects in the Database and we need all 4 objects to paint the UI. Assume a 100ms latency for UI to API calls, a 50ms latency for API to API calls, and a 25ms latency for the API to get the data from the Database and respond. Lastly, let's assume that the client can make up to 4 concurrent calls and all threads are available.
The time taken with both approaches is illustrated below:
Whoa... what just happened! It is actually faster to assemble objects in the UI for this scenario!
Of course, this math assumes that you don't make multiple database calls from one API route (getComplexObjects) which is another best practice that avoids a lot of heartache as explained in my previous blog post: One Database call per API route
Let's see what happens in several scenarios.
|Number of database calls||Available threads||1: UI composes the object||1: API composes the object|
|16||8||250 ms||250 ms|
|32||8||600 ms||400 ms|
And there you have it. Composing the object in the UI starts taking more time only when you need to make a lot of database calls to paint your UI. This should be a trigger to revisit your UI page design anyway, since 32 calls to the database to paint a single page is not really performant in either scenario. In fact, it is still better to compose the entire page in the UI since you can start painting parts of the page as you get data from the Database.
Based on the above reasoning, I believe that complex objects should be composed in the UI layer. Since the UI only makes calls to basic building block CRUD APIs, this approach also eliminates the problem of multiple APIs interacting with the same database object and API route proliferation caused by composing custom objects for each UI page.
I would love to hear similar or differing opinions. Let the discussions begin!