Why Service Oriented Architecture Doesn't Work Internally?

I know I can hear a bunch of clicky keyboards starting to clack as tons of readers are composing a response on their Model-M keyboards to tell me I'm a complete moron. Well, that might be; but before you do let me at least make it through my point. I guess I should add one caveat here; I'm not saying that SoA doesn't work in certain ways. Instead, I'm making the point that using SoA for your internal applications does not work. I'll start by explaining my current situation where we are converting to SoA and how it is proving a complete failure for our internal software. Next, I'll explain why SoA does not work and how the problems SoA attempts to fix has been superseded. Finally I'll explain a better architecture which does contain SOME SoA for external clients and allows for faster response time for applications.

As promised, firstly I will explain my current situation. So we have a MySQL data storage on the backend and up to 100+ applications pounding on it at once. Our MySQL data storage is sharded quite a bit making up 63 hosts, 33 of which are replicated in a master-master replication scenario. There is an average of about 166 databases on each of the hosts. That's a ton of databases; and a ton of data on the backend. On average there are about 10,000,000 queries per day over all the databases/hosts. Now many of the databases can be normalized further (or normalized for the first time) and can be streamlined to actually take advantage of how the MySQL could do better.

Now that we've begun to understand the architecture, let's look at how SoA is being implemented. We're beginning to abstract many of the old database queries into REST-ful service calls. One of the architectures that is one of the newer and "better." This sounds all good and dandy; so let's examine what SoA is supposed to accomplish. SoA is implemented to abstract the data layer; this allows the data layer to change without having to change the applications which are talking to the database. This includes performing DDL or DML changes while not affecting the applications.

So why is it that SoA is failing for us? It's fairly simple, we are trying to emulate our MySQL queries into a midtier application. So why is this bad? Easy, we aren't in the business of making a SQL implementation; nor are we in the business of making high performance SQL implementations. Why is this the case though? Well if you think about an SoA, many times you will need to build an XML or JSON output from the MySQL rows returned. And in some of these instances we need to also perform some other validation or extra data extrapolation. So when we pull down 30,000 rows our service layer might choke because of memory requirements. Well, how about if we just return the first 300. Well we need to sort. So this means that the sort will be done in the SQL but how do we get more than 300? What happens if we don't want to sort by the default sort? You can see how this quickly becomes a giant mess and you begin to create a DSL (Domain Specific Language) build from a DTD in order to do exactly what SQL has already done. But now what if you need to do a count? Oh there is no count? No, instead you need to perform your REST-ful get and then count every element. But how much data was processed in order to receive a single integer?

Let's look at the count(*) example. Our ms are is a simplistic example. We'll assume that our

Step	Time in ms
Read all elements into SQL memory	0.01
Send all elements to mid-tier	0.01
Sort data on mid-tier	0.02
Transform mid-tier results to XML	0.03
Send XML to client	0.02
Count all XML elements	0.03
Total	0.12

Now let's assume that the query is for an InnoDB engine on MySQL which means that a count(*) must count every row in the table.

Step	Time in ms
Read all elements counting as we go	0.01
Send result to client	0.01
Total	0.02

Clearly this is not effective and we should probably look at another example to see when it does work. Let's assume that our client wants all records (200) in a table containing tuples that are 120 bytes per record.

Step	Time in ms
Read all elements into SQL memory	0.01
Send all elements to mid-tier	0.01
Sort data on mid-tier	0.02
Transform mid-tier results to XML	0.03
Send XML to client	0.02
Total	0.09

Now let's look at this if we just queried the database directly.

Step	Time in ms
Read all elements into SQL memory	0.01
Send all elements to client	0.01
Total	0.02

Wow, looks like there is quite a large difference here. This is actually one of the many problems that we are facing attempting to do an SoA. We are also facing the issue of memory needs. For example, let's assume that we have our 200 records at 120 bytes per record. This comes out to 24000 bytes. Not really a huge deal here. But what happens if we have 100 clients that are performing these same types of queries. Except each one is asking to search by a different variable. This means that caching gets us nothing. So let's add another 100 clients. So now we're at 2.4MB of data that is being served. This of course doesn't include the extra XML overhead that needs to exist to send to the clients. It doesn't include the database handles etc... But we can see that if we increase the number of records from 200 records to let's say 500 records; we can increase the storage needed to 6MB. As we continue forward, given a steady rate of record creation we can see that we will eventually run out of space as we try to cache our records while trying to provide fast access to data.

Our issues stem from these exact issues; we've seen our service layer choke and die due to client load. Hundreds of queries per second looking for hundreds of records per second cause these services to choke. Not because the services are written poorly; but because they are not designed for high performance like an SQL database such as Oracle or MySQL is. Now let's look at the decoupling aspect. So now we're going to look at some of the concepts that SOA is supposed to fix with connecting to a data storage directly.

Many proponents of SOA say that the biggest thing you get is the decoupling of the data storage from the application. But let me ask you this; how often are you changing your data store? And even if you are, let me ask isn't this the point of JDBC, to abstract the SQL connection itself? Not to mention that if you do, you'll be updating your service layer anyway. So you're not actually saving anything here. Well, when you make DML changes let me ask, what do you need to do? Update the application layer to take advantage of the new fields. If you have an SOA, what do you need to do? Update the Service layer to take advantage of the new fields. But in the case of the service layer you can provide default field values such that the application doesn't need to be updated on DDL deployment. But what if we used default fields in the database? Don't we end up with the same functionality?

Now here's where we get into where, in my opinion, SOA really makes sense. The SOA can do checks in the data store in order to ensure that the data is. This makes complete sense right? We can ensure that our data is always valid. Unfortunately this same functionality exists in a database; this is the purpose of things like domains and triggers. Again, this functionality already exists in the data layer and is designed to perform faster than anything we can program in. But what about being able to combine tables together and show data in a concise manner while pulling underlying tables out from eachother. An example is maybe we have a customer table, and it contains some information that we want to bring into another table. If we implement SOA, we can just query both tables and return a data type that continues to look like our original table. But data stores have this taken care of as well; they are called views and they allow this same type of functionality.

So what is a better architecture? Clearly connecting directly to the database is not a good option in all cases. More specifically you don't want to have a Web UI that is doing manual database queries from some AJAX client. While that might be true; what you really want to do, is to create a library that contains all of your functions that perform the appropriate database queries. All of your applications (including your web-app) should include the library and make the calls directly. Note, when I say web-app I am talking only about the back-end of the web-app not the front-end HTML/JavaScript portion of the web UI. Now at this point, you can take that library and wrap it into its own service layer to allow external clients to perform queries as they see necessary. The idea is that you don't decrease performance and UX for your application (and subsequently your clients) but provide some functionality externally to other clients that may want to roll their own UX. This provides the high performance required since you have the functions that connect directly to your data storage and yet you have the ability to expose whichever functions you want externally. This also means that you can update the service layer at the same time your application is updated. So where do AJAX function calls exist in here? Well this is where your SOA is necessary; you need the SOA to allow for external connections.

The idea is simple, you treat data requests as either internal or external. If they are internal requests; then you include your library and perform the query through the built-in functions. If they are external requests, then you only allow a query through the SOA. Here are some examples ofeach type.

Internal

• In-house application
• Service layer itself

External

• 3rd party internal applications
• Clients who want an API

I know it seems counter intuitive, but sometimes we have to step back and ask why a specific architecture is better; not just assume that it's better because it "seems cool" or "because everyone else is going to it."