tisdag 24 juni 2014

Microservices - a code problem solved by making it a a CM problem.

Microservices is really a buzzword, as in "if you look up the word "buzzword" the explaination is "microservice"". It's a code encapsulation problem solved by making it a CM, which is really just another problem and really not a solution since CM cannot solve code problems since it's not what CM does. It's like saying that REST is a service and not a protocol, because if that's was true you wouldn't need any code implementing the service... eh resource exposer. It's like if you said "honey, could you take out the trash?" in Swahili but it really meant "the weather in north east will be rain" just because you said it in Swahili.

If any starts talking about REST is the shit for solving all your problems, or how you deal with versioning, be afraid... Very afraid.

P.S if anyone starts talks about API management with some product doing that, run... It's ok if you scream too D.S


tisdag 17 juni 2014

Enterprise architecture is fine but redundant

Using EA for describing your business or how your system work is a hazardous thing to do. You can't use EA or any of the modeling concepts EA use because the code is dictating the result. And because the rapid change of code, EA is doomed since it is not modeling the truth any more. Besides a line of code could change the whole EA map and if you don't know where to look you cannot make any decisions about your system. Because how the system is built and where the relative location of the code and the function it represents resides in a system matters this is what you have to track to make sure that your system is even close the the EA model.

Making decisions from a EA model is extremely risky, as I've seen this bring down CIOs just purely because no one checked or understood the importance of the code. Simply modeling part of a system as an entity of a system is not enough and making decisions from this is, most of the time, fatal. If you have bought a system which does the a specific thing, for example a database or system which keeps information about your customers, you cannot treat them as a black box. As soon you do this, the rest of the system is screwed, and particularly each change to this system is doomed to increase costs exponentially. The location of the functions of the system is extremely important but to actually know if a system is an entity you have to look at the code.

The paradox of a system is that if you don't know what the code looks like and get the context of the functionality you won't be able to make decisions about the system. And that means if you cannot look the code you cannot make those decisions. Even if your treat a part of the system as a black box you still have to know how the code is aligned and where the functional chunks of the code is populated to make the right decisions.

The reasons for EA is a bad thing is the thing it tries to solve is in the details of the code and as long those aren't considered your EA map won't be a good baseline to make decisions. If you have architects which are never even close to the code they will supply you with bad intelligence and a decoupled view of your system. And they will cause the system to deteriorate into a system which is extremely expensive to maintain or change.

fredag 6 juni 2014

It's damn hard to write software

It's damn hard to write software. There are too many variables to track for writing good and successful product. Even the notion of product is different, sometimes it means a new product from scratch sometimes it's a redefined product, perhaps a product which is split into other products. Sometimes the product you are building is the wrong product, or based on something assumed being right but was the wrong choice. Perhaps the product is so old there is no way of tweaking/building/repairing it so it will be a good product but the customer is too afraid/blind/have no idea/clueless/careless/reckless/"hopefully-you-can-fix-it" to acknowledge that. Sometimes there will be some business case describing some fantasy world where the ability of changing the system is either estimated by an ant or the business haven't even taken things into account that there are a system actually doing the work here. Sometimes you have customers you can ask if you are creating the thing they want, sometimes there are no customers to ask, sometimes the customers doesn't even know what they want, or understand what they need or understand what they are getting. Sometimes the customer is some other system and the true customer has no idea what that system does or even do.

You have project managers which have no idea how to do the things you are doing and how important it is to make sure you are able to do what needs to be done. The same project managers only speak timelines and resources, understanding only the next release and only half understanding the customer or stakeholders. Project leaders with no technical knowledge make decisions they cannot even comprehend since they don't know how to code or they haven't event looked at the code. Most of the project leaders are either strictly religious to some belief of process and/or have minions which notoriously upheld sit, or you have the psychopath which does everything by his or her book, no matter what.

You have architects which are, most probably, the only ones in the whole system which are decoupled, and with reality. They do talk about decoupling and systems but have no idea if it's actually true because they are not in development. They draw fancy diagrams about things which should be like how things should be but they are not even close, so they are just doing drawing things. You have some architects which have no idea what your system really does but still comes up with solutions which just messes thing up more. You have enterprise architects which are even more remotely from the system but dreams up some  un achievable goal, aiming for the stars but satisfied if you end up in the tree tops. Problems are both the goal and the material the goal is decided upon is not even close to the reality since they are not even close to the action. There are also strong religious beliefs making up their decisions because of that (*cough* REST).

You have management which only thinks of their next promotion, and thus they just need to make decisions which are just enough so it will look good to their management. The management just over your management makes des icons about things they cannot make since the information about which decision they should do are in the code. They most often have no idea that they are not in charge anymore since every decision is ruled by your system. It really doesn't matter what the management wants, it's the system which has to be convinced.

You have developers which just want to go home for the day, bored at work, disgruntled. You have developers which want do a perfect job but there are no time. You have developers which doesn't understand what they are doing or why. They don't understand the current technology they are using or language. They are swapping between different languages which makes their code less efficient because it's hard enough with context switching as it is. You have developers which works best in a closed room but they are out with everyone else. You have developers that doesn't talk, doesn't share or even care. There are developers with beliefs they cannot prove, they have beliefs which are true to them but irrelevant to everyone else. They are developers which believe what they are doing is art, and in that case everything that applies artistry should apply to their coding and because of that flame wars starts and things turns ugly.

Then there are the system which hosts the products. If you are lucky there are no previous system and you are building it, but then you probably have to deal with politics and religious flamewars. Otherwise you have to wade through bad decisions, all those quick fixes which bleeds the system until the system is no more than duck tape, because some managers just didn't have the balls to do the right thing. You have to write code with technology which is either outdated or not suitable for the things you try to do. You have to integrate with a system which some suit have bought because he believed the sales guy that sold the system to him, and it turns out it doesn't do what he suit thought he needed or it doesn't work with your current tech stack and somehow your competence suddenly is at stake. You have to integrate with a system which is outdated and even if you applies every tech known to man to make it scale it won't. Or you have to deal with some tech consultant which has "the new next super tech which solves all your previous and new problems" and it still won't fit in with the current technology.

Luckily none of this have happened to me.


tisdag 3 juni 2014

SOA is most probably dead at your company.

There are many ways of implementing SOA, now latest buzz word for example is micro services. Many are using some sort of enterprise bus, some says that REST is the way of implementing SOA. Problem is that neither is enough for being even capable of being SOA. It's because they are only dealing with protocols, and protocols are never services and when talking about services the means by reaching it, even REST(whatever flavor) cannot act or even be a service. Even some notation of not using services, whatever that might be, like just wrapping data to JSON, is begging for trouble.

If your goal is to write a sustainable, maintainable and flexible system it will never be about what protocols you are using. It will never be about which application server you are using, or if you are using micro services and particularly never if you are using an enterprise bus, or using some sort event driven distribution. It will and will always be about when you are doing transformations.

Now for example you are using a enterprise service bus with an application server you have failed to being even remotely close to deliver a SOA system. This is because you are not keeping track of your transformations, and fundamentally spread your functionality over several abstraction layers, just by using an ESB. Are you trying to aggregate other services with an ESB you are still creating stronger coupling by just using it because you are not adhering to those interfaces which are defined by either service/resource.

You have just begun your journey to a incredibly much more expensive development, much more entangled system and not creating a flexible system. Everything else is just sales talk. The only way of creating a flexible system is by WRITING it in a way so it won't entangle into a big ball of mud. No external system will or cannot fix anything just by adding it as some additional layer. It will look perfectly good in the simplest cases, particular in a PoC, but in all scenarios and over time you will end up with a more complex solution which will create a less flexible solution as you would without it. If you are using a product for "managing" your API's (WSDL's) or anything which describes your service, you are by just doing this, creating a non-flexible system, because you are not using your abstractions correctly. Using an ESB for anything else than translating a protocol to another protocol is abusing your abstractions and cause your system to turn into a ball of mud over time.

So why is it important to keep track of your transformations? We'll this is because this is where you are providing the interface for your data. But when doing this you must transform your functionality as well if you are going to use your abstractions correctly. Whenever you are transforming data 1:1 and there are some notion of functionality present, directly or indirectly, you are abusing your abstractions and creating a worse system. Which is ALWAYS the case when using an ESB with an application server, other than when transforming from one protocol to another (and that's only because you have no other choice because the application server doesn't understand the protocol).

I don't know when it became some sort of standard when protocols became services. Protocols to me is how you talk to a service, by such means, you could talk about decoupling, ie you have decoupled how you would talk to the service. But a protocol cannot decouple the service from the client since the protocol is not the service, thus the contextual meaning of whatever data you receives from the service cannot be decoupled. The only thing you can decouple from the client is theimplementation of the service. But that will not be done simply by putting a WSDL infront of the service. This also applies to REST, since this is just a protocol a swell, a means of how to reach your resources whatever they be. However the service still needs to define what the resource means. And that will never be done by the protocol.

söndag 23 februari 2014

Software industry

There's something wrong in the software industry. To me it almost looks like when someone comes up with a new religion and there are several people coming up with ideas how to please the new deity. Schools or profets emerges trying to teach the "right" way of pleasing the deity, although in software industry it seems like original focus of deity has emerged into a second more profane deity, the "customer". A number of different rituals are cooked up to mask the fact that they are either not knowing what they are doing or add more scenery to the process to mystify their work or it's some combination of both. It almost looks like a scene from Monty Python's Life of Brian.

All these different "schamantic"(if we do it according to this process, the project will be a success) processes modeling seems to forget the underlying system. It's like if programming is not the issue while building software. Even TDD is not a guarantee that your project is a success and that's a process which at least have something to do with the code. None of the processes are really considering the system and how the system will react to those processes. I've seen Scrum based processes break down since the system doesn't adhere to the principals of Scrum, and no matter what you won't be able to fit in any sort of iterative process in those systems.

As an example is to have three components, A, B and C which depends on each other and a change is affecting all three systems. Now the quirk, system C only has one guy who knows how to code it. Add to it that there are several changes needed to be done at the same time, and things starts to break down. It almost looks like a thread deadlock.

Most of today's processes may it be lean or anything else, forgets the system and what it needs to function accordingly. It's pretty naive to believe that every part of a system could be developed according to some process which is applied throughout the whole development. The most scaring part is that the belief is that the more process buzz words you are using the better chances there are you are succeeding with your project. The system will act as any other employee and needs to be governed. And as human, we react differently to management as a system would.

I think there's a belief that management is ruling systems but it's just the opposite. The system, large and old enough will manage you and there's no way of dealing with that other the way the system is built. If you want system to be governed by an agile strategy, it has to be build as an agile system. But just because you build a system with agile methods it doesn't mean it will be an agile system.

måndag 11 november 2013

Deliverables

Deliverables, what it is really? I've seen so much horrific code delivered by "senior developers", I could almost cry. Is it really their fault? Is it because they are not familiar with the current industry which they are trying to develop some functionality a specific customer wants? Or it is the customer's fault for not being specific enough? Is it the platform they are developing for is inadequate? Is the time frame too short to actually deliver something which is just a bit better, or even great? Or they are just not good enough?

One particular thing is that if you have consultants making the deliverable they, too often, develop like they did on their last assignment. Or like a jack-of-all-trades developing a back end service like it was some front end webpage. The customer, is happy since the application works, but the code is bad. Should I care as long as it works?

I watched a video with Java Champion and rock star Adam Bien where he says "I don't understand why architects always insist of putting things in different projects." Well for a small projects it might not be necessary but for large projects this is absolutely fundamental because there are so many people are working on the same codebase. The code starts deteriorating into the infamous ball of mud. And it really doesn't need to that large before the effects of the ball of mud starts showing. Even the tests starts deteriorating into testing that the method invokes (Unit tests are a different topic, and some of you might scream "code coverage"). Also when starting out you have to layout the fundamentals for maintaining your code since you won't have the time to fix it later.

One thing in common of these deliverables you inevitably write stuff for deliverance, keeping release dates, not maintainability. If you are developing a shoot-and-forget project (you don't need to pick it up and maintain it) I guess that's a feasible way of doing things. Although if you are ending up maintaining it and expanding it you have to be really careful with deliverables or you'll end up rewriting it.

Most companies ends up in this situation. It starts out with a rather major refactoring of the current system to adapt to new expectations or business needs and it starts to ripple through the whole system, and soon you are in a black hole you can't see the end of costing a magnitude of money and time. I've just seen a such thing where the project is coming to an end, but the functionality is just halfway and has to be "maintained" to finish, if ever. And that's one of the problems, system never gets finished but somehow we always act like this is the case.

I believe the agile manifesto is utterly rubbish, however a great idea. I do believe in incremental development, however it requires a huge amount of knowledge to do it, and in the real world you are not getting people which fully understand the whole complex reality of the systems interacting. The problem is really: When solving the problem at hand, whose problem are you solving? Are you solving your programmatic problem, the system's or customer's problem. Also if you are doing a change to a system, how much work should be done to satisfy, not just the "business value" but also the system? With the change you have just done are the code needed of refactoring? Will you do it or will someone else do it? Most of the times no one will do it because there's no time or money to do it. Most of the times it ends up building a intersection without any connecting roads. The intersection works but what about the rest? The most important thing, as I wrote in my earlier post, your company is as flexible to change as your system is and you'll probably end up with the refactoring anyway.

Business value is a misused term, since your systems are your heart of your company and systems must be built for tomorrow. Your system is business value, not the current product you're currently creating. Your company's future relies on that fact, developers, projects managers, architects, management needs to understand this.

The purpose of a system

I talked to a friend a few weeks ago about why his former company didn't succeed with their IT solutions, even with a huge workforce and a big budget. He said that some other rival company did so much better with less budget and people, ergo they have to be doing something better. The rival company is about a quarter of the size of my friend's ex company.

We argued a bit about it and then he said "I'm tired of IT when it becomes some sort of self-worth entity in a company" meaning that it was consuming so much time and effort just doing "nothing". He said "They [the rival company] just changed their system with little effort, from a terminal based consoles to modern GUIs." My guess is that they had wrapped their old system with a GUI. I know that the rival company doesn't do anything if the change hasn't been through an rigorous screening process, even before being developed. They are not even close to being an agile (not as in "agile development") company.

I spoke with a former CEO of a company and he told me that they had invested a huge amount of money in a purchased system to automate their distribution. After the budged being spent and the transition failed they gave up and started over to develop the same thing from scratch and ended up in the same situation as the purchased situation. A product no one wanted or could use. The first case the product wasn't satisfying their needs because the product couldn't adapt to their process, and with the self developed product the budget was heavily underestimated to build whatever process they needed. ie simply ran out of money. They started out too big and failed to reach the goal.

I think the the main problem is the idea of IT being just a tool, particular when the systems are automating your company's business, and the leadership regards it as "just a tool". IT should be a tool and simple things are, but immediately when you start out incorporating your business into a system, the system becomes your company. Your company is as flexible as your tools. As an side effect, if your tools are depending on each other to function the harder it is to change, so just buying more tools won't do it. So to create a flexible company you have to create a flexible toolbox.

When using a tool you can only do whatever the tool is designed for. If you buy a rocket it won't serve as a car, and vice versa. You could build a rocket car but you would have to make trade offs, and its not even certain that the technology is invented to build an effective rocket car.

As an example: Excel is a tool, great for different "business" things. A person can create models and number crunchers quite easily, but even when you start out with filling in the first cell with some value you have to maintain this spreadsheet. The more complex the spreadsheet it is the harder it is to maintain, and the more of the business' information represented by the spreadsheet the harder it is to change. The more people that uses it the more general it has to become, since you want more and more functionality out of the same set of base information. The more people using it the more dependent they become of the tool. As more and more business information is put into the spreadsheet more different parts of your business relies on it. Also at some point you will find yourself with a something you depend on but can't do that new thing which would be so critical to have.

If you decide to automate a process into a system you have to really think of the following parameters:
What am I replacing by automating the business process (what would I have to have instead in form of manpower).
What would be the cost of not doing it.
What would be the cost if I throw away the system (how important is it).
How much will it cost me if we decide to replace the system with another system (are we going to be flexible enough, because this is going to happen).

However these are really hard questions to answer, but I think, they are really valid. I'd say to the first question is that you replace a significant amount of people doing the same job (people can always do the same job as a computer, but in most cases a lot slower). So lets say, I get a system which would replace 50000 people that could cut down costs. The missing problem here is that the system is dumb and can only do what they are told to do. Also that you lock your business with the weaknesses of that system. If it can't do it you either have to buy another product which might do the thing you want and merge those two system, or replace.

Systems can't do everything you want it to do. If you want something out of the ordinary you have to do it your self. The smaller the building blocks needed to create that system you might find some product which does some part of it, but you still have to develop most of it.

And in most businesses what they are doing is quite complicated. A human is a lot easier to "reprogram" although could do what ever they see fit according to the situation, as a computer won't (humans do mistakes, computers don't). You could regard this as, if considering the 50000 people as a system, the endpoints of it is a lot smarter than the endpoints (programs) in a system. And in regards of the third question same applies for the 50000 persons, wouldn't it hurt the company if you fire all of these people and you'd loose a significant amount of your business knowledge since, they outline your business. If not, don't buy/develop that system.

A person is always replaceable, 50000 persons is not, it would probably bleed your company too much if you try to replace all of those at the same time. The same thing if you would just pull the plug on a system, your company will probably not survive.

The same applies to your system. Its relatively easy to write a system. Its really hard to write a modular system, most of the times people say its modular but its really not since there are no (current) ways of knowing if it is. SOA is a good start but its not a guarantee. The recipe for an agile company is to have a system which is always built for tomorrow, e.g it has to be modular, contained and restricted. There has to be a strategy of always change and to be able to do it quick. Even so this is incredible hard to achieve and everyone has to realize that the system is your company and when you have a system, your company will not be more flexible and market responsive than your system is.