JAXenter: Your talk at the DevOpsCon 2019 is about big data architecture finding its way into the public cloud and the involved strategic problems — security is often mentioned as an obstacle. Is the attack surface in the public cloud really that much larger?
Bernd Rederlechner: This question, for me personally, is within the range of “Cloud Paranoia. Every system is more attackable as soon as it gets an address on the Internet. It supposedly helps to have it within your own room and that I personally know my own admins.
Studies show, for example, that a quarter of all attacks come “from within” or, at least, have assistance. I think everyone has to do more security today – whether in the cloud or not. Most of the basic mechanism for encryption, defense, and monitoring are available at the touch of a button in the public cloud. But with self-construction it is not so easy to quickly set up such mechanism by yourself. Often it fails due to a lack of specialists. The responsibility of developers is also growing: Instead of shifting security to infrastructure, it belongs in software design. Switching to the cloud is a good opportunity to do this homework right away.
It is also important to choose a cloud provider you trust, even in critical, technical, or political situations. Personally, I think it is important to listen to what your gut says, and not to just run after the herd.
JAXenter: Was siehst Du bei einer Migration von Big-Data-Architekturen in die Public Cloud als größten Nutzen?
Bernd Rederlechner: As the name suggests, big data is all about handling large amounts of data. This generally means that you also need a lot of resources to evaluate and store data. The public cloud has the great advantage that I can get additional capacity if necessary or release it again at short notice. I have just accompanied a project with a truck myself. It makes a difference that there is a ban on driving on Sundays. On Mondays, however, many people leave at the same time and I really need a lot of computing power. Resource-intensive systems with variances in data traffic over the day or week therefore benefit from the dynamic consumption model of the public clouds, especially if you work with automatic scaling mechanisms.
So-called streaming architectures process large amounts of data quasi as a “fly by”. There is not always a lot of saved data, depending upon the use case. It is then more a matter of triggering reactions quickly and reliably. The core of such systems is based on highly available and scalable messaging infrastructures. It is no fun to set up a professional messaging-backbone by yourself – not even if it is a self-constructed one in a public cloud. A streaming service as a completed PaaS service, on the other hand, avoids a lot of surprises and headaches in this regard.
As soon as large amounts of data need to be stored, the costs of persistent storage come to the fore. In the public cloud, there are very inexpensive storage variants for large amounts of data. Unfortunately, the cheaper the storage is, the slower it becomes. Architectures that take this into account run very cheaply in a public cloud. We call this “architectures with data economy”. In my own data center, I usually only have hard drives and backups available (at least as commercially interesting variants). Therefore, architects rarely integrated a multi-level data economy into their architectures. Unfortunately, however, this means that a simple reinstallation of such legacy systems in the public cloud — without modernization — hardly benefits the architecture.
JAXenter: Can and should I use platform services, or should I avoid vendor lock-in? Or is there a clear fall back or exit strategy, and if so, what does it look like?
Bernd Rederlechner: The “boring as a service” principle applies for me: “Build your own services where customers or end users of the system sense a clear advantage or perceive a real difference to the competition! Use managed PaaS services wherever you do the same boring stuff as everyone else!” This puts me knowingly in the hands of this dangerous “vendor lock-in”, i.e. the problem that I cannot just change the product later on. But in all seriousness, if you look at investments in modern corporate landscapes, the fear of a vendor lock-in doesn’t seem to apply to all providers and IT products anyway… and by the way: Has anyone ever changed a product without much effort just by simply switching-over?
Good architectures have always distinguished themselves by splitting an IT solution into smaller, decoupled units via defined interfaces. If you proceed domain-driven, i.e. professionally, and not technology-focused, these interfaces, like bulkheads in a ship, limit the necessary changes when switching a platform service. A strictly professional modelling of a service quasi limits the underlying technology dependency. I show this with an example in my talk. In the case of a switch, the interface has to be re-implemented, but in most cases you do not have to “screw around” all over the system in an uncontrolled manner. Switching is possible, but not as freely as you might have imagined it.
In the end, you must not exaggerate the need for protection against vendor lock-in. It is an illusion to believe that you can predict vendor lock-ins anyway. Who would have thought that Spectre and Meltdown would make us realize that using Intel processors is a classic vendor lock-in?
JAXenter: To what extent can automation help to remain cloud-independent?
Bernd Rederlechner: An interesting question. It shows that the connection is apparently not as obvious as I had thought it to be. In the past, automation consisted mainly of small helper scripts that made the administrator’s life easier. In a DevOps world, the bar is much higher: At the push of a button, automations create entire system environments or let them die — and absolutely reliably, because otherwise the entire Continuous Delivery Pipeline comes to a standstill. So we’re talking about more complex software — and that’s why automation should also have some architecture. Architects have learned from domain driven design that an anti-corruption layer is a good tool. This decouples me from system aspects that I cannot control directly. More tips and details would go too far — that’s what my presentation is for.
To me, cloud independence does not mean that I can switch clouds at the touch of a button, without having to think about it. That has never worked either. But an anti-corruption layer as a kind of predetermined breaking point in my automation helps me to be able to connect to and use another cloud as needed with manageable effort.
JAXenter: But in the end it is mostly the money that counts. What advice can you give developers to convince their managers about the benefits of the cloud?
Bernd Rederlechner: Currently, it is enough to tell your manager that company xyz (preferably a competitor) has a public cloud-first strategy. Jokes aside, in my experience, there is always an emotional component to cloud discussions: It might be the fear of loss, because I no longer own my IT, or due to evil hackers at whom’s mercy I am now, defenseless in the cloud. There, evangelizing hardly works..
If you can’t convince with words, you better convince with facts. This can be a simple cost calculation, which shows that my reference test environment in the cloud is much cheaper, because I switch it off in the evening, and on weekends. It can also be a test installation of my application on a public cloud, which adapts its resource consumption to the respective needs by means of automatic scaling. What also works is any kind of improvement to a business application, where I would have to wait a long time for the delivery of the infrastructure or even for a strategic initiative in my own data center. Managers usually come along quickly when a business improvement is immediately possible, and without too much extra cost.
JAXenter: How do you evaluate the use of cloud services in terms of the industry? Is the cloud the only way to go or will hybrid systems remain the reality in companies for a long time to come?
Bernd Rederlechner: I actually believe that in the next ten years only big companies or special security fields will afford a hybrid infrastructure. Wherever the infrastructure has to be paid for directly or indirectly by my business, a self-powered landscape reduces my profits more than a landscape in the public cloud.
It is usually this point where the outcry of the IT responsibles (or even some of my own colleagues) comes into play. But I can justify that: I am not referring to the obvious cost comparison of investment vs. rental costs in the public cloud. I am concerned with the hidden costs, and disadvantages of “waiting until I can use a component” when building a system. Digital solutions require many modern software frameworks that need to run redundantly, robustly, and quickly. It is not the mere installation of the software that makes me wait, but the creation of a platform with comparable high operational quality standards, just like in a public cloud, takes time.
If the quality is lacking, then unplanned work delays the delivery progress. But especially digital business models often only pay off if you are the first one on the market or at least follow the front runner quite quickly. So why risk the business and forego a service at the touch of a button — just so I can pet a server in my own computer room? Not to mention the question of what is more complicated: the maintenance of a hybrid or the pragmatic migration of the “remnants” into a public cloud, and the handling of a single cloud.
