Stateless application architecture is the current de facto approach, right? Not necessarily. Stateful applications communicating over TCP sockets can and are successfully built even in today’s age of cloud-native applications.

Our guest on this episode of DevOps Chat is Chris McCord, creator of the Phoenix open source software, and architectural engineer at DockYard. Chris came up as a developer using PHP, Java and Ruby. When he learned that WhatsApp was built using Elixir, running on the Erlang VMs, Chris was intrigued. What’s needed is a developer-friendly web framework for creating web and mobile applications.

Chris started the development of the Phoenix Framework in 2011 and saw adoption pick up in 2014/15. Fast forward to today, Chris is an integral part of a vibrant community of developers using Phoenix to create web-oriented, mobile, embedded and real-time applications that can support large transaction server volumes with less code.

Join us as we talk with Chris McCord about Phoenix, why he created the open source project which today expands into Phoenix LiveView, and the Phoenix Frenzy developer challenge (winners to be select in early-to-mid October.)

As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

Transcript

Mitch Ashley: Hey, everyone. This is Mitch Ashley with DevOps.com and you’re listening to another DevOps Chat podcast. Today I’m joined by Chris McCord, architectural engineer at DockYard. Our topic today is Phoenix, web framework for Elixir. Now Chris is also the creator of Phoenix and he’s also an author, so I think we’ll have plenty to talk about today. Chris, welcome to the podcast.

Chris McCord: Thanks for having me.

Ashley: Great to have you. Would you start out by introducing yourself? Tell us a little bit about what you do and also what DockYard does.

McCord: Sure, yeah. So as you mentioned, I’m the creator of a web framework called Phoenix. It runs on the Elixir programming language which runs on the Erlang virtual machine, and I’m sure we’ll get into what all that means for maybe some of the audience that isn’t familiar, but essentially what I do apart from building the framework is work with DockYard to build out full stack projects for our clients, so that’s anything from Fortune 100 companies to startups, and we use Elixir to power all the solutions for these clients. We also do design and UXD, but as far as my focus it’s really focusing on backend systems and building out scalable platforms.

Ashley: Certainly a big need these days with all the applications that we’re creating and need integrating and scaled. So let’s jump into your work and I’ll start first with what is Phoenix and maybe a little bit of background on Elixir for folks that don’t know what that’s about, and Elixir/Erlang.

McCord: Yeah. Phoenix is kind of like the de facto web library of framework for Elixir, and so the quickest synopsis is Elixir is a programming language which runs on the Erlang virtual machine. So Erlang is a language that has been around for over 30 years now. It was actually developed in the ’80s by Erikson to run telecommunication systems, so they kind of built this niche language to solve a very specific set of problems in the ’80s, and meanwhile the programming community at large focused on object orientation, single-threaded languages. Fast forward to today, where we kind of have Morris Law in effect and we’re starting to add more cores to keep up with the CPU speeds, all of our traditional languages have kind of suddenly hit a wall where it’s like suddenly we can no longer count on our programs doubling in speed every year. We have to add multiple cores, but most languages weren’t made for multicore age.

It turns out the specific niche set of problems that Erlang solved in the ’80s actually maps perfectly on the multicore age, so I think we had this kind of nugget of innovation in this Erlang virtual machine kind of sitting there largely unnoticed, and then the creator of Elixir José Valim came in in 2011 and kind of recognized this innovation and building modern language with some kind of features that programmers are used to basically in these mainstream languages and build a language on top of the Erlang virtual machine. Elixir was birthed on top of this niche virtual machine, and not that Erlang has had its own successes—you know, we’ve heard about like WhatsApp with their $20 billion acquisition. They’re running 10 million users per server and I think they supported that with like 30 engineers total.

Ashley: That’s amazing.

McCord: So yeah, that’s kind of the heritage and has what allowed Elixir language kind of takeoff much faster I think than what a typical new language would be, and then of course I was able to come in and literally stand on the shoulder of giants and build a web framework on top of Elixir, which gives us all these amazing, scalable features.

Ashley: So exactly the metaphor I was thinking, we all stand on the shoulders of giants. That’s one of the great things about our industry is what’s old is new again and we build on the successes of the past. Talk a little bit about why did you embark on creating a web framework on top of Elixir? Were you working on a project and saying, “Hey, I keep building the same code over” or “I have friends that have the same kind of need, maybe I should go build something”? What was sort of the inspiration for creating Phoenix?

McCord: Yeah, so for me my background is like Java and then PHP and then Ruby, but for the previous, let’s say, six or seven years, before I got into Elixir I was doing Ruby professionally and I was loving it. Doing a lot of Ruby on Rails, for folks that are familiar with that framework.

Ashley: Ruby’s a great language. Love it.

McCord: Yeah, and I had a lot of happy clients. And as kind of the web evolved to this more real-time connected nature we had more and more requirements, and the more and more types of applications I wanted to build were real-time connections, showing live updates on a page, doing things that you would expect to be able to do or that we expect to be able to do as users on the web today. I was hitting walls with Ruby. I was trying to make libraries to support kind of real-time applications and things just were very brittle and weren’t scaling well at all, so that’s when I started looking around to say like, you know, if I could pick a language to write kind of like real-time applications in, what would I pick? And that’s kind of where I discovered Erlang and WhatsApp had just been acquired, so I kind of saw how they were running millions of users per server and that’s where I said, “How are they doing that?”

I want to be able to do that. And that’s when I discovered Elixir, and the creator of Elixir, José Valim, had come from the Ruby community so I kind of remembered that, oh, you know, he went off and created this language; maybe I should go see what he was doing. That’s kind of what sparked my interest and then I fell in love with Elixir and I said, “The only thing I need to do is write a web framework so I can build all my client applications in this,” so that’s kind of what started the whole thing.

Ashley: Isn’t it interesting, if I remember right, in Ruby I think libraries are called gems or packages of libraries, so that experience there, so how do I something like that in this new language that you’re working in, and it will eventually lead you to creating Phoenix. What a great story.

McCord: Yeah, exactly. And it was kind of under this promise of platform, right? So I guess with any technology you kind of take a leap of faith and I dove in reading the Erlang WhatsApp, millions of connections per server, and then assumed that we could do that. So long story short, two years—or once we reached 1.0 in Phoenix we did benchmark and our real-time layer we benchmarked up to 2 million connections on a single server.

Ashley: Wow.

McCord: Yeah. We were only limited by—we had to spin up like 45 servers to send that amount of traffic and we just ran out of connections, so we weren’t limited on the server. We could’ve pushed it further. So it’s kind of been an interesting history, kind of taking this leap of faith, assuming what you read about a technology is true and assuming that you can actually easily achieve or it’s achievable by you personally and then kind of went through and showed that we can scale to millions of users, so it’s been a really interesting and fun ride so far.

Ashley: Very nice. Well, and if I understand you started this around, what, 2013? I think the first sort of public or production-ready release 1.0 was about 2015, so it’s been in the community for a while. I’m sure it’s matured since then. What’s the state of Phoenix today?

McCord: Yeah. So we’ve been 1.0 since 2015. I think my first commit was 2013. Elixir was started in 2011. It didn’t really hit any kind of usage till 2013, so once Elixir went 1.0, which has been—it was before Phoenix 1.0, so 2014, maybe? So we’ve seen pretty stable and large adoption since then in the language and the framework, so Elixir itself and Phoenix are still on one-dot whatever release and neither the language or Phoenix, we don’t—we have no plans for a 2.0. I mean at some point in the future there will be a 2.0 but we don’t have—we have a couple deprecations but there isn’t like something on the horizon where you’re like, OK, now I have to rewrite everything. So our goal is basically a stable core at the language level and the framework level, and we’ve had companies come in and we tout like a Bleacher Report. It’s one big example we always point to because a lot of people know who they are, so that’s Sports News website. They have like 80 to 100 million visitors a month and they’re built almost entirely now on Elixir and Phoenix. So we’ve had companies come in and build on top and seen really great success so far.

Ashley: Are there any particular kinds of applications that are particularly well suited for Phoenix, like a web application like you mentioned a sports site? It might be delivering a lot of video content or photographic content, or is there other applications more transaction oriented like processing payments or whatever it might be through a web framework? What’s the best kinds of apps that really take advantage of Phoenix?

McCord: Yeah, so I mean the most basic answer is anything connected to a TCP socket [inaudible due to crosstalk].

Ashley: That’s a lot of stuff. [Laughs]

McCord: Right.

Ashley: That’s a lot of things. [Laughs]

McCord: Basically, all of the above to what you said as far as anything web-oriented, anything with like notifications. So the Bleacher Report started with their adoption of the push notifications to users and then Pinterest has not all their stack, but the notification layer of Pinterest is written in Elixir. So we see a lot of companies doing kind of these message broker style use cases, but I think anything connected to a TCP socket, whether it’s a web server, orchestration layer for your platform, stuff outside of what we would consider like the web is still gonna be a great fit, but I think the biggest adoption we see so far is more web-oriented just because I think that’s the biggest market share developers, right?

All of us had to do something web facing but we have a broader reach beyond that and that’s not even to talk about—we also have—Elixir can be used in the embedded space, so you can actually do Elixir and the entire Erlang run time into like a 16-megabyte Linux image and run it on an embedded device and you have all of these benefits. So it will work equally well on like a Raspberry Pi Zero up to a 100-core server, and that’s the most exciting thing for me.

Ashley: Interesting. Is Phoenix more well suited to a stateful kind of application or stateless or does it make a difference really?

McCord: Yeah, so stateful applications are—I mean it’s the thing that we’re able to do that no one else is able to do, or do well, as far as these mainstream frameworks. So we can still do stateless HDP just like anyone else, and we can do that incredibly quickly and scalable, but the reason that brought me into Elixir was we can actually build stateful applications with connections that stay live. You know, we don’t have to throw away these stateless connections and try to fake some state, so that’s basically why I built Phoenix. We call our real-time layer channels. It’s similar to like a Socket.IO. So basically it’s an abstraction over web sockets, and it’s what I wanted to do and that’s what brought me in, was the ability to hold these real-time connections.

Anything we build today is ultimately a chat app, right? Whether it’s Twitter or Facebook or all these things we’re trying to build. You need some kind of real-time nature, you need to be able to push updates to users, even if it’s just to show like a notification count on the header bar of a website. That’s way easy to do if you actually had a conversation going with that connected client versus trying to have like long pulling or take it over stabilization ___.

Ashley: Well, it’s interesting with the cloud and that’s really kind of brought stateless application design architecture to the forefront, but if I recall, Erlang’s really well suited to doing stateful kinds of applications and large volumes of it, so it sounds like that’s part of what attracted you to this environment.

McCord: Yeah, exactly. Yeah, the whole stateless, serverless approach is interesting to me because it’s like it’s the opposite of what we’re able to excel at in our platform, and what we’ve seen is like—maybe just one example. Someone was paying $16,000 a month on AWS Lambda and they rewrote that in Elixir and they’re paying like $40.00 a month now.

Ashley: [Laughs]

McCord: So I do think that, yeah, there are merits to these stateless platforms, but I think it’s under the guise of what the alternatives were prior. So if your platform alternatively was mostly stateless then having someone else handle your stateless concerns and scaling makes a lot of sense, but you can now build a stateless. I think we kind of turned that whole idea on its head and we say, “Well, we can just run all this on a few servers and it’s going to scale for the lifetime of our business,” then that’s way less complexity and way less than these other solutions.

Ashley: Interesting. Now I think you also just recently concluded a talk. Was it Phoenix LiveView? Was that the name of the conference? I think it was the challenge that you did.

McCord: Yes. It was ElixirConf.

Ashley: ElixirConf, that’s right.

McCord: I talked about Phoenix LiveView, yeah.

Ashley: What was your talk about there?

McCord: So I presented LiveView last year at ElixirConf and it was kind of like a proof of concept, and then we’ve been working on it for a year and I really presented how far we’ve come in a year. So last year it was kind of prototype, had some promise, and this year it’s practical use cases and the kinds of efficiencies that we’re able to get and even how we’re able to beat kind of traditional single-page applications using Phoenix LiveView.

Ashley: Cool. There was this thing called Phoenix Phrenzy, too. I think it was a challenge that came out of it, correct?

McCord: Yeah, so the Folks at DockYard put their free time together and helped get this contest site together, which is really just a contest for the community to put examples of Phoenix LiveView together, kind of to one compete with each other and also to kind of grow a broader awareness of the platform outside the community. So yeah, Phoenix LiveView allows you to build we say rich, real-time applications with server-rendered HTML, so you don’t have to write any JavaScript yourself and you’re able to get kind of these interactive applications in much faster time and much less complexity than what you would probably be used to. So the competition is a way for people to kind of showcase their work and what’s possible.

Ashley: And how are the winners selected of the competition? Is there a board that you’re a part of or how do you decide this is the best of the best?

McCord: Yeah, we have a handful of judges. Quite a few members of the Phoenix core team are judges as well as a few members in the community. So I think we’ll just go through and kind of vote on our favorites and whoever gets the most votes wins.

Ashley: What’s the prize? Is there a grand prize other than the recognition?

McCord: Bragging rights. [Laughter] Yeah. In the future we will look into prizes. If anyone’s ever run a competition like this you immediately get into some legal waters trying to award prizes, especially with different geographic and age restrictions, so really it was a matter of trying to make sure we can do a competition without waiting months on the legal process, but maybe in the future. But for now it’s just global recognition.

Ashley: Well, I think there’s something to say for attaining some Phoenix street cred, so I think that’s [laughs] probably worth it itself.

McCord: PhoenixPhrenzy.com—Phrenzy with a P-H—is available and anyone can join, and like I said, for me it’s just really neat seeing people build things with LiveView, so I think people should give it a shot if they’re at all interested.

Ashley: One of the great things about the opportunity to talk to you, Chris, is it’s very cool to be able to talk with someone who had this idea, created this framework, wrote books like “Programming Phoenix” and “Metaprogramming Elixir,” et cetera, that you’ve created. You have this contest speaking about Phoenix at ElixirConf, so you really kind of built a career around doing this but it’s something that you kind of birthed, what, five, six years. That’s got to be a great feeling of satisfaction for you.

McCord: Yeah, it’s been a very interesting ride. You know, when I started this I didn’t have any plans for where it was going. It was kind of—I always wanted to be involved in opensource from the altruistic side of things where I was using opensource libraries and frameworks so I felt like I was taking and not giving back. So when I got into Elixir it was more like a passion project, right? You know, I wanted Phoenix to be successful but I had no idea if Elixir itself would even be successful, so as things kind of took off, yeah, it’s just so interesting to see it grow, and especially from—you know, there’s a whole other conversation in here that we don’t have time to have, but as far as like burnout and once Phoenix became popular and people were using it and then I suddenly was like, oh my gosh, I have to maintain this, right?

Ashley: Look what I created. Uh-oh. [Laughs]

McCord: So anyway, so it went from—you know, fortunately I was able to find a sustainable path from there, because I was working full time at work and then working full time at home on open source, so I think had I not found DockYard where DockYard hired me to spend the majority of my time on open source I would probably be living in the woods somewhere. So fortunately I was able to find a sustainable path to opensource development. That’s a very hard bridge to cross I think for a lot of maintainers.

Ashley: It is.

McCord: So fortunately for me I was able to, yeah, cross that bridge successfully and it’s been mutually beneficial for myself, the company, and the Elixir community.

Ashley: Well, kudos to DockYard for doing that, for finding the kind of synergistic relationship with you to allow you to do so much work on it and continue that work. Speaking of that, what’s the future of Phoenix? What do you see coming down the road in the next six to 24 months? What’s gonna happen?

McCord: Yeah, so I think LiveView, it’s probably where the biggest hype is because, like I said, you can build an equivalent feature application for some single-page app but with probably like 10 times less code and 10 times faster to market because you don’t have to write all this JavaScript and we can do it in a scalable way. So I encourage folks to check out my keynote at ElixirConf if they want to see more about how I can make those claims, but we talked about our optimizations. So I see that kind of being at the forefront for the next year or so as far as what’s happening in the community.

Ashley: Do you see—

McCord: But beyond that for the framework itself we’re—go ahead. Yeah, I was gonna say beyond that we’re going to add like telemetry and metric integration in the framework so you’ll be able to at a glance see what’s happening in your system, so that’ll just be built into the framework where you can see—you know, if your system’s slow you can exactly pinpoint what’s going on at the framework level and your application code level and can diagnose issues. I think that’s one of the bigger features that’s coming that is not yet in place, but outside of that I think we’re going to grow organically around kind of this real-time applications, whether that’s with channels or Phoenix LiveView.

Ashley: Fantastic. Well, definitely an enjoyable conversation and I feel like we could talk for a couple more hours. I’d love to sit down and grab a beer with you sometime, but thank you for being on the podcast. It’s great to have you here, Chris.

McCord: Yeah, it’s been fun. Thanks a lot.

Ashley: Well, to our audience, you’ve listened to another podcast and we’d like to thank Chris McCord who’s, you know, not only an architectural engineer at DockYard but of course creator of the opensource project Phoenix and author, thought leader, and great problem-solver, so it’s great to see good things happening with Phoenix. So, thanks for joining us, and of course we’d like to thank you, our listeners, for joining us today. This is Mitch Ashley with DevOps.com and you’ve listened to another DevOps Chat. Be careful out there.