The big one is that OpenStack is about building and operating your own Rackspace; this is not something IT organizations can even hire for let alone carry off. The idea that they could/would/should was purely aspirational.
The others are that it's basically a mess - no two OS deployments will look the same - and that it has neither an operational advantage nor too much of a pricing advantage (50% for RHAT) over VMware - which is better integrated and much, much better from a admin experience and debuggability standpoint.
I looked hard at leveraging OpenStack for the service we were building but the underlying code was often cringetastic and somewhat naive. At some level if you run software as a service you can work around poor code quality - something friends from AWS have emphasized - but you can't throw it over the wall and have other people without large engineering staffs there to help run it - which doesn't work in the small. VMware is the opposite approach - the entire model and practices evolved out of arm's reach software sales and support. So naturally it works better.
As a small competitor to Racksapce in 2010, designing a replacement for our (then) 8-year old VM structure, Bytemark bet against Openstack from the 1st announcement and spent years building something the-same-but-different.
It wasn't _just_ that Openstack was unreleased and talking about foundations and making grand promises without any production-grade code. It was just that we could never work out which version Rackspace was actually running for their actual customers, and the details of how to actually _deploy_ it and make it work seemed enormously fiddly compared to VMWare, which was where they were taking aim. I expected them to say "now running on Openstack, 100%!"
Also the design seemed to mirror AWS, as if the only answer to their dominance was to ... copy every facility they were producing exactly?
We had a more definite vision of where VMs should go, and thought it was a bad plan to aim at "Amazon, but smaller!". In particular we really really wanted live migrations to be a part of our platform - where we really really cared about uptime of individual VMs, and wanted people to be able to upgrade them on the fly. Plus, y'know, for a hosting company, being in control of, and having opinions of our hosting platform was what people paid us for!
I see people asking whether Openstack is actually running anywhere very public, and have read some pained war stories. So it still feels like the right decision, even before this announcement. Plus it seems like live migration in Openstack is still an exotic and difficult option, whereas that's been our standard practice for years.
There are plenty of problems running your own hosting stack, but I like that we can solve our own problems on our own terms, and tie software decisions to hardware or data centre decisions that we make at the same time.
I eye up Kubernetes & Ganeti every few months, but they keep reminding me that if you've got the expertise, and a specific purpose in mind, you can usually build something that fits your purpose more closely.
(though the second half of that sentence will probably be written on my self-carved tombstone)
The use case I described - Skyport's cloud managed secure servers - needs super high quality, self-recovering embedded code that made no assumptions about the network being high quality or reliable.
I think the thing people - and especially the OpenStack guys - don't appreciate is how terrible it is to (a) lose a workload or (b) require someone visit the datacenter (which may actually be a colo in another state or hours drive away). Having to fall back to some sort of terrible insecure management like IPMI or a dedicated mandatory management network (of questionable security), etc. is just not viable.
Systems and infrastructure architectures need to address a few things that really matter - error handling, continuous self monitoring, state compression and linear, systematic self-recovery - and while some OpenStack components handle this (ceph is pretty great except for a few things around access control and security) the whole doesn't handle them well at all. It's not enough to log a message (or worse, just log an exception stack).
I find your expectations to be unreasonably high.
The self correcting system is a fiction and will always be a fiction. The indefatigable infrastructure that corrects the errant member system with 5 9s is a fiction and will
always be a fiction.
We (scientific research institute) are currently operating a relatively small commercially supported Openstack (Liberty release) installation next to our HPC infrastructure and so far it works quite well.
Of course, because we are using Openstack from a commercial vendor, the installation/provisioning is quite straightforward (although there is quite a bit of complexity, if you want to do a lot of adjustments). So far we didn't have any big troubles during operations and our users (researchers) have started to use it (creating VMs for specific applications).
We will hopefully upgrade soon to the Newton release which will enable us to provide SaaS (Murano) and CaaS (Magnum) to the users. We will probably also evaluate Openshift on top of Openstack.
The biggest pain points is the integration with our existing HPC infrastructure (specifically the parallel filesystem/storage).
We haven't really looked much at the code except of debugging some authentication issues, when we tried to integrate keystone with our core IT's AD, so we can't comment on that.
Additionally, the official openstack documentation (the commercially supported one is quite superficial) is often very confusing and it's very hard to find proper CHANGELOGS for each of the openstack components.
Also we really don't like (or rather despise) openstack's issue tracker (Launchpad).
It is very hard to find any existing issues, when we encounter some problems.
At our facility, we have an OpenStack-based private cloud that we worked with Sardina Systems to put in place. It is production and we have not had any problems with it.
We have also undergone upgrade from Mitaka to Newton with zero-downtime and completed within a day (I understand that zero-downtime upgrade is a capability specific to Sardina FishOS). We will be upgrading to Ocata in the coming months.
We have found that in cloud, it is critical to separate the Operator role vs Consumer role. OpenStack is primarily Operator-facing, who are in turn providing the capabilities to serve certain Consumers. OpenStack enables the Operator to flexibly provide virtualized compute environment requested by the Consumer, while the Consumer does not care how the compute environment is provided by the Operator. With this view, lots of things become clear. Granted, not everyone in the OpenStack world have this view, but we are sure glad that we do from the early on!
Our deployment is not the largest, but we aren't small either (several hundred hypervisors). We are an organization of several 10's of thousand people, though the user base at the moment is still in the hundreds.
A number of comments here mentioned that they had live migration problems. We have not had these problems. It worked from the very beginning for us.
Our view was that we would work with a vendor on OpenStack, rather than trying to string together the parts, in the same way that we don't string together the parts that make up a Linux distribution. FishOS makes it simple for us -- and that's before looking at operational tools in the product.
I have not been involved in the commercial terms, but my understanding is that commercially we have TCO that would be hard to match otherwise.
the article mentions other big players pulling out or cutting jobs/funding related to OpenStack in recent months in addition to this latest move by Intel.
Is this related to some endemic problem with the technology, big egos within the project, dominance by another outside player or what? Cursory searching doesn't reveal any "big thing". Maybe it's just dying a death by a thousand cuts.
As one of the original authors of OpenStack and one of the top contributors, I may have a unique perspective. I think the OpenStack community made two related mistakes:
1. We thought that private clouds were generally valuable. In 2010 it seemed like everyone was going to have a private cloud. It turns out if you are a midsize business, having a rack of "cloud" doesn't really offer you much benefit over having a rack of managed vms. You still need to pay someone to manage that rack. The vast majority of the benefit of cloud comes from having a ton of workloads, which means a public cloud or a huge service-provider sized business.
2. We focused on community building by supporting all use cases. The best way to build an open-source community is to bring everyone in. The community grows fastest that way. Unfortunately, it also means that the product quickly becomes an unfocused frankenstein that is decent at everything and good at nothing. Projects like docker and mesos are suffering from the same problem.
The end result of these mistakes OpenStack is good for certain use-cases. It does really well in large companies that need a public-cloud like environment to manage their infrastructure and can hire a team of people to manage it (e.g. comcast, verizon, e-bay, wal-mart).
I don't know that the exodus is due to endemic problems so much as the market finally waking up and realizing that public cloud is the future.
Openstack created a lot of possibilities for the way it "could" work or "should" work, but almost no methods where it does work.
The problem with Openstack is it had too many moving parts, and no standard/reference implementation. It was destroyed by it's own grandiose goals. The project wanted to say 'yes' to every permutation of software. Making it a disaster from an operational standpoint. Everything promised, but nothing verified.
Agreed. Many moving parts, and every vendor could (and did) add their own variant of the moving parts, to integrate with their hardware/software/... And of course they then try to sell that and not the "standard" implementations, so not enough engineering work goes into those and the core components. It's a "collaborative" project with stakeholders that try to compete where they can...
Visible also in things like the installation process (which for ages didn't have an "official" answer, so you had pay one of the big vendors to do it if you didn't want to try automate it yourself)
I second this many times. The APIs were so inconsistent. I had to go through many hooks just to get the data I wanted. Openstack is nice, but Openstack community tried to create an open AWS. The problem is, everything gets so modular and too many components to deploy just to get Openstack working.
It's a PaaS for IaaS foundation. Unless you have the resources to line up and support Openstack, I recommend just build on AWS/Google/Azure/Rackspace/Heroku/DigitalOcean, you name one.
I wonder if big customers such as Bloomberg ever regret / think of moving to public/private IaaS.
Openstack was supposed to be a vendor based consultant bonanza. It wasn't designed by or for the users or customers, if it was designed at all.
Instead it was a vehicle for hardware vendors to sell compatible product and consultants to build something they'd have to service forever afterwards. (Yes this is a cynical and half joking take, but it's also serious)
So what went wrong? Welll, If you're not going to build a product that teams can actually figure out how to use, then you need to have Cisco or Oracle level sales talent behind it. OpenStack didn't have that, and maybe couldn't have it because there wasn't one boss company behind the product.
Like I mentioned, there is a trade-off between the usability and required Sales Power. As far as usability goes, even consultancies that were directed to use and sell OpenStack solutions couldn't get their people to figure it out. A consultancy based technology has to be coherent enough that a junior associate can reasonably start billing hours for it. OpenStack will stymie a close-knit team of veteran "devops" people— And I'm talking about people who actually know systems administration and development, not a buzz word ninja.
The problem is that "consulting bonanza" with open source software is really hard to do. It is really, really hard to draw a line between what is custom engineering and what is a non-upstream patch you will have to maintain forever. And that's why Red Hat's mantra is to do everything upstream first, even at the cost of sometimes delivering features later.
Everybody got that completely wrong.
Open source works, but you still have to understand how to engineer it. So far, companies that succeeded are mostly following an "open core" model where they control the core too (Atlassian, Automatic). Red Hat is the exception, and with Red Hat putting a lot of effort on upstream OpenStack development no one could pull off open core for OpenStack. Mirantis has plenty of non-upstream code and last I heard wants to sell SaaS to work around that.
Plus, no one devoted the right amount of engineering to the lower levels of the stack (at least QEMU and libvirt). You cannot seriously expect to partner with vendors for guest operating systems without knowing all of the stack---this was something else that Mirantis and others missed.
So the problem was not that OpenStack was bad, it was that everybody thought open source is magic. But it isn't magic, and if it looks like magic, it means you don't understand it.
I figure Intel's participation was hinged on supporting a growing fleet of competitors to AWS, who would bid up chip prices. I guess that isn't happening.
IMO, OpenStack was sort of a consortium effort to compete with VMWare, AWS and Salesforce, but it's operational model is closest to AWS. Nearly all the big enterprise IT companies tied their rafts together hoping to stave the flow of customers. It doesn't appear to have worked.
RackSpace is advertising AWS migration and consulting services on podcasts I listen to. Not exactly a stellar testament for their own cloud product, but I think they still participate.
HP's strategy is a bit haphazard given the recent corporate splitup. On the plus side, the HP/HPE split seems to have allowed HPE to ditch all the weird www# URLs. They retired their public cloud, but still sell hardware that can run OpenStack at least.
Dell bought EMC, and proceeded to shut down their OpenStack offerings in favor of VMWare, which they own a substantial fraction of.
IBM is in a continual process of downsizing its hardware divisions in pursuit of higher earnings per share. It seems their big push is cloudfoundry, which seems to be more about containers and k8s.
So why are they bailing? I'm guessing:
- it's way harder to hire support staff for OpenStack, than VMWare
- AWS reduces capital costs and upfront investments
- Customers on existing solutions aren't prepared to take advantage of new opportunities
- Many of these companies have existing product lines they don't wish to disrupt
You missed Red Hat, who seem to be the only company having major success with OpenStack [1]
Most of IBM, Intel, HPE etc have thrown in the towel and now offer their own services on top of Red Hat OpenStack.
OpenStack has now found itself beyond enterprise, and now being the de-facto platform for NFV running mobile networks, and I guess Red Hat are becoming the winner here as they are so used to supporting an OpenStack 'type of' infrastructure for large bodies such as banking, telco, health etc. When you consider Red Hat are already large well established contributors to all of the layers of the OpenStack 'stack' such as KVM/QEMU , libvirt, the kernel itself, + overlay networking tech such as OVS, and now DPDK, you can see why they are well positioned to support and run OpenStack clouds.
SUSE folks on twitter have also been highlighting that they are still hiring, even in the wake of the addition of the HPE OpenStack team (or what was left of it).
Given all of the inter-dependencies between OpenStack services and lower level Linux constructs and software (OVS, Ceph, QEMU, etc.) those who were already used to distributing and managing these already turned out to be the ones who were best able to make the combination work.
That said at least some folks who were working on OpenStack and related technologies at Canonical are among those now looking for new gigs:
Thus while cloud remains on their priority list it is not immediately clear how big a part OpenStack will play in that going forward for Canonical versus solutions that run on the reigning public clouds.
(Disclaimer: I work on Red Hat in the virtualization team).
Canonical seriously lacks expertise in KVM (+QEMU+libvirt), and you cannot take that for granted when you have a customer with VMs crashing that is asking for a fix.
Those who have down-voted the above almost certainly do not know the situation on the ground.
FWIW, I recently had to explicitly inform the Canonical Virt maintainers (of which there seem to be very few) about which patches that ought to be backported to fix a bug in one of their libvirt packages that was seriously affecting (i.e. preventing from patches being merged) OpenStack upstream CI environment.
The said bug had fixes already available upstream, and are straight backports with no conflicts. No one bothered to do the "unsexy" work of backporting & cutting a quick stable build.
Only after pointing out the commits (with help from one of the lead upstream libvirt maintainers), and posting them on a LaunchPad bug, did the Ubuntu maintainers stepped in to backport the said fixes.
> It seems their big push is cloudfoundry, which seems to be more about containers and k8s.
Cloud Foundry is built on technologies that predate k8s, but yes, containerisation is the unit of operational currency.
Insofar as Dell EMC has a multi-play, it also includes Pivotal, which is the leading contributor to Cloud Foundry. Unsurprisingly it runs smoothly on VMWare, as well as AWS, GCP, Azure, OpenStack and I forget what else.
Disclosure: I work on Cloud Foundry on behalf of Pivotal.
Yeah, the first advertisement I heard for Rackspace's consultancy service for AWS was a bit of a shock. Having heard it, I'd assume that they're pulling away from their own cloud-based service offerings at some point.
Just conjecture on my part, but I would have serious reservations about that part of their service catalog right now if I was vendor shopping.
Their own cloud isn't mentioned on it. They've obviously been putting their eggs in the consultancy basket for 6 months now,
Hell, even if you go to cloud in the menu, they're only advertising other clouds: https://www.rackspace.com/cloud You have to go through the 'Infrastructure' menu to find their own cloud offerings.
I'm wondering how long before we get notice we need to migrate away from them due to impending shutdowns.
Wherever it says Openstack, that's Rackspace cloud. All the cloud stuff under Infrastructure is powered by Openstack, running on Rackspace hardware in their data centers.
Where did your source come from to say Dell EMC have been shutting down their Openstack offerings? All offerings were still there for Newton, Pike is just around the corner and I havn't seen any indication the same drivers won't be there for it too
I don't know about today, but a few years ago I tried to set it up and found it to be an uninstallable massive ball of twine and chewing gum. It was shockingly clunky. I can't imagine running something like that in production. I suppose it'd be good job security for a lot of sysadmins.
It hasn't improved much, at least not of three months ago. Automating the rollout of any particularly is a Herculean task and even doing it by hand is onerous.
Devstack works. Sort of. Poorly. Everything else is pay-a-consultant territory.
I just moved into an openstack development job, and I'm struggling. So many technologies and complexities you have to keep in mind, even within a single project. I'm hoping it comes together soon. I have not provided a ton of value yet just sitting here learning, and breaking so many things.
It is a lot to ingest. Can you elaborate on the OpenStack development you are doing?
OpenStack refers to a number of different projects (Nova, Neutron, Horizon, etc). I would advise getting a high level understanding of how Nova, Neutron, and Cinder/Swift interoperate. Then drill in to Neutron. As with every other cloud platform I've worked with, most of the IaaS complexity lies in Networking, in my opinion. Neutron, and SDN in general, is a fairly complex topic depending on your experience with network architecture and administration.
I am doing Monasca development within an Openstack cloud running on Kubernetes and built with helm.
So for example a development environment I use is a docker-compose environment building Monasca. That environment is a work in progress, so half the time I'm fighting that environment just to make a proper change.
Then from there, I don't have Horizon, so I have to use command line apis directly to set everything that I wish to use.
The whole thing has just been a challenge, every part I've had to learn in some respect. The actual code I've changed in Monasca, has been the simplest part so far. Validating it's correct, and testing it has been the far harder challenge.
Monasca also being a bit of a challenge since it's several different repos of services working together, instead of a large one akin to Nova.
A large part of it is that those who can make money out of OpenStack are doing so, and those who aren't are doing the math and moving on to the next thing.
Take a look at the sponsors for KubeCon and Dockercon and consider that many of these same companies were those originally trying to catch the OpenStack wave, and then ponder how many of them will be extracting themselves from those projects in 5 years (or at least lowering their investment, which as we see here is generally perceived as extraction).
Prominent folks in the Kubernetes community will tell you that it's different but the recent conferences sure have a similar vibe to OpenStack summits circa ~2013 when the marketing really started to overtake what the technology could deliver.
>Is this related to some endemic problem with the technology,
I believe the decline of OpenStack is inherent in the type of technology. I don't believe that type of "infrastructure plumbing" software lends itself to high-speed high-quality innovation in the "open source" model. Some reasons:
1) it's boring infrastructure software: Because it is software for "data center" infrastructure, it's not glamorous and interesting enough to attract programmers who don't interact with that world. Other successful open source projects like LAME MP3, ffmpeg, Linux kernel, etc have programmers that can experiment on laptops and ordinary users that can download and play with it.
2) no abundant source of high-quality code contributions: The enhancements you want contributed into OpenStack should be provided companies running complex private cloud operations. They would battle-test the code and that knowledge would trickle down into the open source contributions.
However, a company that's facing the prospects of a complex private cloud would more likely choose Amazon AWS or Google Compute platforms instead of tackling OpenStack.[1] This removes a subset of potential OpenStack contributers. OpenStack is so complex to run/customize/maintain that it requires building up an internal staff with skills similar to AWS/Google/Facebook's datacenter engineers.
If the company does decide to run OpenStack, it's likely they won't contribute back any source code. (Same as Linux situation where most users of it don't contribute source code.) This removes another subset of potential contributors.
Probably the best "private cloud operating systems" are developed by Google/Facebook/Amazon/MS for their own datacenters and those 4 companies are not contributing to OpenStack. Wal-mart has sophisticated datacenter operations and they may be the largest Openstack user but their contributions to OpenStack do not match the new innovations of AWS mentioned at re:Invent[2] or Google CloudNext[3]. Yes, Red Hat is a sponsor of OpenStack but their internal cloud operations are not stressed like AWS/Azure/GCP with customer-facing challenges nor do they manage IT complexity like Wal-Mart.
Sometimes, commercial versions of plumbing software do lose to open source projects. One example would be Microsoft's Dryad "distributed computing" losing to open source Hadoop even though MS had a 7 year headstart. However, Hadoop+HDFS is still not as fast or reliable as Google's internal distributed mapreduce. (Maybe the idea of buying a hundred expensive Windows Server licenses to satisfy a 100-node Dryad cluster contributed to the market ignoring it.)
Based on history of how certain open source projects fall behind proprietary counterparts (such as OpenCL not as cutting edge performance as NVIDIA CUDA or OpenGL not matching MS DirectX features[4]), OpenStack seems on to on a similar trajectory of "not as good as the proprietary alternative".
At this point in time, the "private cloud OS" space is moving very fast (possibly faster than flavor-of-the-month Javascript frameworks!) and the open source development model is not able to keep up.
"If the company does decide to run OpenStack, it's likely they won't contribute back any source code. (Same as Linux situation where most users of it don't contribute source code.) This removes another subset of potential contributors."
From my perspective, companies that decide to run Openstack do contribute code back, but the process is so long and arduous to get code accepted that it's not hard to see how groups would just give up and just merge upstream to their own.
Core reviewers have to accept your code, and they are on no timeline to do so. Months will go by without a review.
>such as OpenCL not as cutting edge performance as NVIDIA CUDA or OpenGL not matching MS DirectX features[4]
I think that's a slightly different reason because OpenCL is just a spec with no real teeth. NVIDIA/MS have a pretty tight grip over the market both in h/w and s/w, and creating a CUDA like spec does nothing meaningful if there is no alternative source for hardware and software. Everything including drivers, libraries, compilers etc. is closed, whether it's CUDA or OpenCL. So aside from some vague promise of portability, OpenCL doesn't really open anything.
The irony on the "high quality code contributions" being that this partnership between Intel and Rackspace was supposed to be a substantial testbed for software development, and a sort of residence program for corporate developers to come up to speed on the code base.
Is there any chance that vendors (like Intel) will care less about the API for accessing their products, as there seems to be more consolidation around API's for running applications like Kubernetes. Intel is a member of the CNCF [1] and seems to be putting money behind it. Am I reading too much into that and is Intel just a big company that puts money in lots of places?
They have an interest in a large server ecosystem in general, with options for running your own infrastructure, because it's a lot more profitable for them to sell server CPUs to many companies that don't have as much options to pressure them on price as e.g. Amazon can, and to get their special features integrated so they can use them as arguments against other CPU vendors. They don't care as much which option wins, so it makes sense to be involved with anything that seems important.
It's probably a little of everything. Intel does put a lot of resources into projects like Kubernetes and other communities adjacent to Linux Foundation / CNCF, but I think they put a lot of money anywhere they see a future for utilizing their products... and I think that gets little abstract, sorta. Or rather, they invest in some things that seem like they're partially related to a lot of their core interests.
Intel's new bread and butter is the server space. They are doing everything possible to make sure they stay on the forefront.
With the slowdown of single threaded scaling Intel knows that performance improvements for consumers in the near future will be increasingly anemic. Servers and ML are two spaces where increasing core counts still makes sense.
I very much agree. Clicked link, read five words, site started screaming some garbage advertising at me, closed tab immediately. It's not even the video that bothers me so much - it's the autoplay audio that causes me to immediately close the tab and never return to a site. Ugh.
I've recently started using this Chrome Plugin to disable autoplay: https://github.com/Eloston/disable-html5-autoplay. So far works about 95% of the time, but it recently became unmaintained. It also does not distinguish between when you click on a video and the next link plays it, e.g. Following a link to a YouTube Video or Netflix show.
Consortiums are inherently unstable. Marketing interests drive wild swings in participation and funding.
Open governance by individuals is much more stable. That stability benefits both individuals (whose stake is not vaporized when they change employers) and businesses (who are more insulated from impulsive moves by big players).
If you just want clustered virtual machines, check out Ganeti[0]. It's not advertised much, but this piece of software hosts most of Googles internal infrastructure (not public facing stuff).
Unlike Openstack, it has a proper scheduler, and lets you rebalance VMs across hypervisors efficiently. Also unlike Openstack, it can restart VMs if it (or the hypervisor) dies, if you've enabled that.
And completely orthogonal to Openstack, it has very strong consistency guarantees. It's not made to start 1000s of VMs in seconds, since each master node has to agree on all decisions, and each operation typically "locks" the involved hypervisor. On the other hand, I haven't been able to break it once in over six years.
Note that it really just exposes an API and comes with a superb command-line client. Some assembly required.
Source: deployed Ganeti with great success at a billion-euro company, moved on to promising "cloud" project which insisted on using Openstack and promptly quit after a year of fighting obscure bugs (and naive colleagues who did not want to try anything else :)).
There are some, but in practice I suspect the real competition comes from the major public cloud services in one direction and from more traditional but tried-and-tested IT infrastructure in the other. So far, products aiming for the middle ground, like OpenStack, just haven't quite found their mark. After this much time and investment, in a market where potential is trumped by pragmatism, it's not surprising that major corporate backers might start to look elsewhere for their future strategy.
We believe OpenNebula is a very good alternative to OpenStack. It's a completely open source product, with many strengths. Very easy to setup and to maintain, without the big mess that is required by OpenStack. Plus, users don't get caught in all the politics (lots of vendors pulling in their direction).
So depending on your use case, some of you might be very interested in CloudABI. See: https://nuxi.nl to solve some of your issues. Ed is a FreeBSD committer and has been working on solving some of the harder problems relating to security and god I hate to use this word but "containers" or "containerization" of applications. If you write your app to the CloudABI spec it will run on any of the supported platforms. The video of his presentation at the 32C3 conference does a much better job explaining it. It's certainly worth your time to watch.
The article just mentions that Intel is no longer funding OSIC, which from what I read is essentially a data center for contributors to use (for free?). I could totally see why Intel would have second thoughts about footing a server bill; I don't think it means they're officially out of OpenStack.
My understanding is that the OSIC program also included funding for Rackspace + Intel OpenStack developers, some proportion of whom will probably be losing their jobs out of this I imagine.
From someone not very invested in the OpenStack process it feels like all the momentum of the recent push of the project just withered away all at once. I am not sure if the container crowd ate it's lunch, where containerized appliances just solve the problem of openstack better.
[Disclosure: I spend a (not too large) _portion_ of my time on the OpenStack Compute project, so adjust my comments for bias.]
Please. You seem to be making the same mistake, that many make on blogs and news websites, of exclaiming "containers are the future!" while both VMs and containers have their place and use-cases.
I'd rather suggest you read the below. (Since I'm linking to a blog, quick background about the author: Rich is one of the long-time contributors of KVM-based Virtualization stack and the lead author / maintainer of libguestfs.org)
VMs have their place, but a lot of what kind of built the OpenStack hype engine - dynamic workload deployment and multitenant-style management - also is what driving the (also partly aspirational) deployment of containers.
The thing is, though, they are at odds. Enterprises don't have nearly the dynamic workload set that people thought but where they legitimately have some is in devops style development. And if you have container management, which you do, then highly dynamic VM management is much less valuable.
I am not really claiming containers are the future, but from someone who is actively working on a network appliance I have people asking me at least twice a month about containers to put my product in (from people who are already managing 10k+ systems and wanting to simplify deployment). I have heard zero buzz about OpenStack except during a SVP drive by about a year ago.
Most of the customer sites I interact with on the high end just end up baking their own system images tied to a SSO solution and just manage it with VMWare/Ansible/something homegrown. On the low end people just don't deal with OpenStack at all and just do it the hard way with a VMWare or KVM style solution. For either side of that equation containerization is a faster value-add.
From the other side what people ask me about cloud integration is more feature-driven of particular clouds than anything else. I'm just not seeing the momentum in the field.
OpenStack is a platform for creating an Infrastructure as a Service (IaaS). So it's running on some physical machines, and manages your physical machines to slice into VMs. There you get to manage / create networks, storage, etc. Basically running a private cloud in your own datacenter/basement/office. Openstack has grown more than just compute. It has a supposedly compatible interface for S3 an object storage system, for example. New components are created after major cloud services like AWS, pretty much. To manage containers or manage applications, you can use CloudFoundry/Kubernetes, but to manage the "infrastructure as a platform" (I want more machines, I want some bucket, I want to run some Lambda function, I want to upload new OS images) you need something like OpenStack. So if you want to build AWS/Digital Ocean, OpenStack is required.
This seems to get back to an interesting discussion I had with someone early in the OpenStack efforts: as a developer OpenStack doesn't directly interest me because I don't care about infrastructure. Where OpenStack had a possibility to win was to provide options for infrastructure agnosticism: if I can build an app that runs "unmodified" on any OpenStack-based infrastructure, that has a possibility to save me potential time and money from having to port apps to/from/between AWS, Azure, Google Cloud, et al (assuming of course that it enough clouds actually adopt).
From that perspective, container solutions are delivering a better developer proposition than OpenStack has yet managed. There are ways now to build container clusters that you can ship in parallel to AWS and Azure with very little code difference.
In that earlier discussion I was skeptical of OpenStack precisely because of its focus on infrastructure first. Without the buy in of being a clone for a specific cloud structure (AWS compatibility over anything else, for instance) or the backing of traditional datacenter/server vendors (IBM who eventually started into BlueMix; Microsoft whose "on premises Azure" is now firing on most cylinders but was announced as a plan early in OpenStack's history), OpenStack didn't seem to have an obvious niche in the infrastructure world. The closest to a niche it might have had in its early life was the promise of application portability between clouds and that never quite seemed to be delivered.
I can tell it frustrates infrastructure folks to hear that containers have been eating OpenStack's lunch, but that is the very real case from the developer perspective. As a developer today, I go for containers and OpenStack is no longer relevant on my radar. Sure I can run containers on OpenStack, but containers abstract away more of the infrastructure and I have less and less care what cloud(s) is underneath the container cluster. When I asked OpenStack people what OpenStack might deliver to me that vision of application portability was tantalizing but never seemed quite finished; container technologies have actually delivered that.
> if I can build an app that runs "unmodified" on any OpenStack-based infrastructure, that has a possibility to save me potential time and money from having to port apps to/from/between AWS, Azure, Google Cloud, et al (assuming of course that it enough clouds actually adopt).
> There are ways now to build container clusters that you can ship in parallel to AWS and Azure with very little code difference.
OpenStack is an IaaS, so just like AWS. Focus on the context. Do you want your own private cloud? Yes or not?
If no, then this discussion can end, because AWS and Azure run on their proprietary IaaS code. As a customer, you request resources from the IaaS layer, and you build your server/platform from that point.
So arguing that container clusters can ship to other clouds with very little change (almost likely writing the APIs to create a container) is unfair in the context of why one would choose container over OpenStack. The purpose is different.
If your answer is yes I am building a private cloud, how are you going to do that with Docker alone? Can you build a software-defined network with Docker? Absolutely not with Docker since Docker is a host-based deployment solution.
What you are looking for is ability to create OpenStack the same way CloudFoundry / Kubernetes are created. You write up a manifest, and the necessary databases and services are deployed to some EC2 machines. In the case of CloudFoundry, you write up a manifest file, describes number of instances, types, credentials, what not, then call Bosh to create Cloud Foundry (will create a pool of app machines, router servers, UAA, etcd etc). Machines are created based on a stemcell, basically an image. You want to create your IaaS based on images. You want to be able to script up a manifest and deploy your IaaS. You want a lift and drop IaaS infrastructure. Container can do that, but it cannot be done simply with Docker. You need that infrastructure abstraction layer to cover up. That's probably what Docker Enterprise Edition might do, but I have not really dig into it yet.
Yes, it is an unfair comparison. I was attempting to explain in my comment why I consider it a valid unfair comparison, because it is a matter of perspective.
As a software developer, do I ever want a private cloud? No. Does my employer? Maybe. Is it my job to tell them how to invest their infrastructure dollars? Quite possibly no, because software development and infrastructure are typically held at arms length. But even when they are not in a "proper" DevOps shop, the ballgame of which cloud is then subservient to developer convenience and how easy it is to deploy software to a cloud and how productive developers are writing software for that cloud.
So yes, the purpose of OpenStack and Container technologies are very different and I appreciate that technically. In terms of real world value to me as a software developer, however, I have platform problems not infrastructure problems. I don't care what the infrastructure is under the service so long as it provides a stable, reliable platform for me to build upon. Containers abstract that for me in a way that solves real platform problems that OpenStack was only ever relevant to me in so far as its ability to once hint at a possible solution to. That's not fair and that was expecting too much from OpenStack at the time, but that's life.
Okay, I was reading it as defending why container would solve what OpenStack was set out to solve, which is the proposition I read from the OP I was replying to.
Of course, I would advise against running a private cloud unless there is a dedicated team of at least a dozen or so. I applaud Digital Ocean for able to survive and make good business from their private cloud. As a developer I totally agree I just want my code to be deployed and that all the appendices are deployed and configured.
Not with Docker alone, in the same way that you can't do anything with OpenStack alone. It requires a lot of plugins. You can use docker/kubernetes with OVS and have a somewhat workable SDN solution.
> if I can build an app that runs "unmodified" on any OpenStack-based infrastructure, that has a possibility to save me potential time and money from having to port apps to/from/between AWS, Azure, Google Cloud, et al (assuming of course that it enough clouds actually adopt).
OpenStack is at the same layer as AWS, GCP or Azure. It's not an abstraction over them -- that's a PaaS. I'm not sure OpenStack was ever seriously intended to work at that level.
I work on a platform, Cloud Foundry, which provides the kind of portability you're talking about. Red Hat have OpenShift, which is another such platform.
Disclosure: I work on Cloud Foundry on behalf of Pivotal.
It doesn't have to be an abstraction; it could have been an API standard for IaaS providers, with OpenStack itself serving as a reference implementation of said API. No higher-level concepts like containers; just saying things like "the VM lifecycle controllers in AWS, Google Cloud, Azure, DO, Linode, Rackspace, some random company's private cloud, etc., should all 'speak Nova', so that any Nova client can control VM lifecycles on any of them." Such providers could expose extra features with their own proprietary APIs (think e.g. the features exposed to the Gmail users through the web-app vs. through IMAP) but the standard protocol would at least be there for everything.
Not that asking AWS+GCP+Azure to commoditize themselves in such a way makes much economic sense, but that's the proposition.
> I'm not sure OpenStack was ever seriously intended to work at that level.
There was a hint at some possibility that it might, especially in some of the hype circles early on. It is of course unfair to blame OpenStack for never actually fulfilling that dream, when that was not its intention per se, but that is part of why today people continue to compare what OpenStack promised versus what containers delivered.
It's frustrating to read people comparing container technologies or worse container schedulers with OpenStack. Apples and oranges.
If you are planning on running containers on premise you need something from managing the IAAS side of things. OpenStack is excellent at IAAS and is the foundation for running higher level abstractions and services on top.
I ran an OpenStack cluster in my house for a few years. The deployment was managed by a bunch of scripts which I wrote and published to help others learn the basics in deploying an OpenStack cluster, sans most of the more complicated networking stuff. Last count, about 25K unique IPs downloaded the images used in those scripts to launch test instances. At one time I held the top 3rd or 4th link on Google for "openstack install".
A few years ago, I leveraged a bunch of methods in OpenStack to build a thing which would launch instances based on Bitcoin payments to certain preloaded addresses. These addresses allowed "templates" to be associated with cluster capabilities and code to be deployed. Payments to that address in Bitcoin would immediately net you an instance, of a certain type, on someone's infrastructure.
The general idea behind this creation was a way to abstract hardware components into a system by which applications could launch themselves and utilize the resources provided in a fair, secure and trustworthy way. It is my belief this model was WAY ahead of it's time, a precursor to the hybrid models we see emerging today, and lead directly to my personal realization that federation of all systems will be a basic requirement for implementing trusted computing in the future. We're going to need it with AI. Not sure how I know that, but there it is, irrationality and all.
Unfortunately, standardizing deployment methodologies doesn't net you federation. Standardization itself doesn't net you trust, unless everyone can agree on the standard, which is out of necessity done in an non-trustworthy way. Votes of a board, for example, aren't represented with fair consensus, unless there's an algorithm behind the votes. Otherwise, votes end up being slightly irrational, because people behind the votes are slightly irrational. Or very irrational, depending on the company they work for.
And yes, federation can be implemented in a trustworthy way by a corporation, such as Google or Amazon, but that requires all people in that federation (a discrete group) agree they will use this thing to implement trust, even though the thing may not actually have trust implemented in a rational way (i.e. by algorithm).
At the end of the day, OpenStack was doomed not because of container tech, or the structure of the board, or who ran the biggest cluster, or because it was overly complicated for simple use cases.
It failed because it did not implement the basic requirement of delivering trusted infrastructure in a scalable and trustworthy way across a broad range of infrastructure, in a wide range of locations, and do so in a way that separates the use of the infras from the irrationality of humans running the infra.
Until something does that, and does it well, we're stuck with Google, Amazon and other provider's solutions. This is also a good rationalization for the continued increase of cloud services by companies and the continued emergence of hybrid models in the future.
I'm at one of the only companies I know of outside Rackspace that's running it in production. We're doing a very poor job of it; we never hit production with any effort to upgrade the cluster in the past three years, so we're still running Havana with Nova Network.
The most likely upgrade path for us right now is VMWare 6.5.
Huh, it's that bad for you? I used to work in the Openstack team at Yahoo, and thanks to the efforts of my colleagues, the first thing we got out of the way was upgrades. One of my friends made Anvil[0] We stayed with the community till the SVP decided it was time to move all of Yahoo to AWS.
I always wonder if one of Openstack's weaknesses was also the same thing that was it's strength, it's openness and need to support everything, as well as remaining agnostic to users. Nearly every company I know needs to register VMs on a local DB of some sort, and has patches on Openstack to do that. But every time I went on IRC and asked if we should think about a nicer way of providing this hook, I'd either be redirected to a different project, or get some rude replies.
I also wish there were some more modern companies behind the project; a lot of the developers were probably under pressure from penny pinchers in the management.
That said, it was a terrific project to start my career on.
So are we. And so is another department in our org.
Frankly, we're doing a piss poor job of it. Migrations don't happen even if manually ran, the compute nodes are already at capacity, even with the meagre VMs we use.
I have kept pushing for Mesos myself. I see it as a good way forward. Openstack may be a technical good body of code... But in the end its too convoluted codebase and many moving parts to be effective in most deployments.
Mesos isn't really a direct replacement for Openstack, but you're probably still right to be pushing for it. If you roll with Mesos you will still need to solve a bunch of lower-level infrastructure challenges: machine provisioning, basic system config management, storage management (if you want persistent volumes), fancy networking (if you want ip-per-container or any sort of network isolation), and so on.
Like Openstack, Mesos brings its own set of new complications.
Unlike Openstack, Mesos almost always increases efficiency of operations once it's in use. Openstack always seems to result in more and more effort spent on just operating Openstack...
Too true that Mesos isn't a drop-in replacement. Far from it. There is quite a lot of complexity as well in Mesos..
But looking at it, there are also less moving parts with Mesos compared to OpenStack. As a test, I set up a Mesos Cluster in 3 VM's. It wasn't terribly hard at all.
But at work, we had an alarm thrown in nagios about critical ram consumption on a node. So I go and do the migrate. It works.... And it doesn't. It migrated the VM to the same node. I had to do 2 extra flags to tell OpenStack to migrate the node properly. That's just ridiculous, and dumb.
And not to mention, but Mesos supports proper handling of VMs and migration. It frustrates me to no end the amount of hand-holding I have to do with anything OpenStack based. Whereas Mesos was "Install, configure, run".. With a large amount of work on Configure.
Not really. Kubernetes is stupidly unstable. When I was testing it a while back, the master processes kept dying for no good reason and leaving half-on processes with no networking.
And then you get to restart everything. Yay.
Whereas if Kubernetes is on something else (OpenStack/Mesos) you at least have control to restart Kubernetes when it shits its brick.
The "a while back" qualifier here is important. Both Mesos and Kubernetes are relatively new projects in a rapidly moving space. FWIW we are running Kubernetes 1.4x with multiple redundant masters in production with no problems. We are also running Mesos/Marathon internally. These projects did have major issues early on but now they are becoming much more robust.
For control we are using CoreOS with kube-aws. It actually works quite nicely as we can scale and upgrade running clusters. The only issue with it right now is upgrading and managing etcd.
I think the whole issue with OpenStack is the overambitious goal to be a one stop from bare metal servers and switches to applications. Kubernetes with Flannel or Calico can take care of the software level networking and provisioning, while there are a lot of other ways to provision VMs for running Mesos or Kubernetes depending on where you are deploying.
I've been in "the industry" professionally since ~2005, and with Linux since 1998 or so. I have a business need for kubernetes and there is nothing at all religious about it. It is simply that we have lots of new applications that need to be spun up and deployed. They might need to scale up or scale down, and we have gobs and gobs of on-premise (read 0 cloud) new hardware. "API Driven" infrastructure is the best way to achieve a better story for the various ops teams, and so we're building out some kubernetes clusters globally for them.
A better 'story'? This is terrible jargon as is the entire storied approach.
There are dozens of ways to achieve the technical outcome you aim at and docker is one. Docker is the less toil|overhead, less flexible|performant, vendor lock-in, learn a new toolset approach. Save your indignation for someone who doesn't know better.
Because running docker (which is the least stable part of my entire infra) on thousands of machines is a good idea when there isn't a sensible way to do orchestration? Totally some startup knows better than google about container orchestration. The same google that literally added control groups to the Linux kernel which sort of invented linux containers.
Spare the drama dude. Just because something doesn't work for you doesn't mean it doesn't work very well for others.
No, we're not. Our environment is small; only about 100 VMs. We're a Debian shop, so no redhat.
No, it's not a 1:1 replacement. It's a migration. It would also be a migration if we decided to upgrade OpenStack, because we'd be going Havana -> Ocata.
However, there are defined upgrade paths for ESX, we already have ESXi/vSphere in our environment, and we can hire someone relatively inexpensively to help us with vSphere if we need to augment our staff. Folks who understand Openstack are not anywhere near as easy to find and pay.
I worked for a company (Nobis Technology Group) that was acquired by LeaseWeb USA; I was a part of the "server automation" team and was somewhat reluctantly "forced out" of the team after the transition (they basically offered to pay me a smaller salary to work for them - do I look stupid?).
The team was essentially dissolved several months later, as I knew it would be.
But for the short period I was there (about 2 years), it was a great place to work, and a high point in my total career. Prior to that position, I had been doing web development in PHP pretty much exclusively. Doing server automation was a completely new space to me.
Soon after starting, our team was tasked with a migration to OpenStack. Since our current infrastructure front-end was already based on PHP, I got tasked with looking into how or if we could use PHP OpenCloud to work with OpenStack. It seemed workable to me; I was able to extend the classes in such a way using namespaces and other techniques so that we could add additional capabilities to the interface that weren't already supported (and there were a lot of holes to fill!), but wouldn't break things if/when we had to upgrade OpenCloud (this was ultimately tested a couple times I was there, and the changes proved flawless - at least on the PHP side).
Ultimately, we had a nice stable front-end that worked well with both our original system (some VM architecture that I forget) and the new "cloud" infrastructure based around OpenStack. In effect, a user could deploy either an actual server (if available), provision a VM (if a server was available), or build a cloud server "system" from a myriad of parts (we tried to support and provide as much access to the OpenStack stuff as we could). We also had a RESTful API for clients to also used (some of our clients resold our services under their own names). Some of the backend stuff was a bit "messy" in how it worked (I won't go into details, but I "authored" a fake "O'Reilly" "book" (really just a front "cover" mainly) whose mascot was dickbutt) - but despite the mess, overall it worked well, considering all the moving parts (where it would tend to fall down - not always, but enough - was when an upgrade to OpenStack was performed).
In short - we were also one of the few companies running OpenStack in production. Our owners ended up selling to LeaseWeb, I left - but the idea was that LeaseWeb wanted to transition things to their API and system, and I honestly don't know what happened with all the work and such I was involved in on the PHP side of things (there was also a point where me and a coworker had to quickly ramp up and learn GoLang to make an interface from Rancher/Docker over to the Nobis API - that was a fun and interesting experience). I imagine that some portion is still running, but who knows.
I personally think that in the right hands and with the right infrastructure OpenStack can be a very workable and working technology. It seemed to work well for the systems we used while I was at Nobis. I don't know honestly whether you could use it to scale up to anything like AWS or Google's offerings, but I think for medium-sized stuff like we were doing (or like Digital Ocean does - who at the time was our direct competition), it can work well - at least as I experienced things.
I don't disagree with you at all. For our uses, it was a stupid amount of overkill specced by someone whose job was the same as his hobby. We don't have anywhere near the need to scale that such a system was designed for, the hardware that was specified was very poorly considered, and no maintenance was ever done on it except for daily care and feeding -- which quickly grew to consume all of the previous engineering team's day.
When the previous engineering team was shown the door, the entire system had no maintenance and no path forward. It's an epic management fail, but ... let's just color me cynically and sarcastically surprised that I used "management" and "fail" in the same sentence.
I have seen several successful implementations in the 18 months I've been working with OpenStack. There are definitely gotchas and low level (driver, hardware)issues you have to deal with. I never thought I would be so familiar with IPMI.
That said, OpenStack is an ambitious product. It essentially seeks to give you a lot of the functionality of AWS, except with on-premise infrastructure. That is a tall order. AWS is a multi-billion dollar effort that I assume has has far more engineers than the OpenStack project.
OpenStack is a complex solution to a very complex problem. Having worked with other commercial IaaS solutions, I will say I hands down prefer OpenStack to the others I've worked with.
If there is a private cloud solution that delivers the same/similar functionality of OpenStack with less admin overhead and more simplicity, then I want to hear about it.
We're running Openstack with Kubernetes and it's working quite well. Lots of challenges though, and we're just exiting a preview phase into production.
I was at their keynote in one of the Openstack summit. I can guarantee that at least the VP that spoke about it knows nothing about openstack, and were just using Mirantis to deploy. They just threw Buzzwords around, one of which was Fuel[0]
Worked for rs in the early 2000s. The culture was pretty cutthroat and it was the chiselers and greasy guys who moved up.
I worked as a dc tech during the .com bust recovery period
and the level of incompetence and the personalities I encountered led me to leave < 6 months.
Any decent SA from my generation can code a one-off virtualization base without too much problem using kvm/qemu.
LXC is a clean in-fit to that. Sounds like this is standard RS procedure. Fleece the idiots, use the willing, and promote the owlshit.
The big one is that OpenStack is about building and operating your own Rackspace; this is not something IT organizations can even hire for let alone carry off. The idea that they could/would/should was purely aspirational.
The others are that it's basically a mess - no two OS deployments will look the same - and that it has neither an operational advantage nor too much of a pricing advantage (50% for RHAT) over VMware - which is better integrated and much, much better from a admin experience and debuggability standpoint.
I looked hard at leveraging OpenStack for the service we were building but the underlying code was often cringetastic and somewhat naive. At some level if you run software as a service you can work around poor code quality - something friends from AWS have emphasized - but you can't throw it over the wall and have other people without large engineering staffs there to help run it - which doesn't work in the small. VMware is the opposite approach - the entire model and practices evolved out of arm's reach software sales and support. So naturally it works better.