Finding A Sane Stack For The Web

Everyone loves sharing their web dev stacks! There's even a whole popular site dedicated just for that.

Rather than just blindly sharing the stack(s) I prefer, I thought I'd write a little bit about my general experiences and goals with web stacks. I'm really big on believing in creating a core stack for individuals and companies to work with over extended periods of time. The challenge, then, is finding what initial setup has both enough flexibility for diverse projects as well as enough existing functionality as to provide very quick initial iteration times.

Starting From The Backend

The language+runtime environment I've ended up using the most is JavaScript with Node. I've explored and tested various alternatives to it, including Python+Django, Go, C#, even Rust. While JS certainly has its problems, I still continue to find it the most suitable option for the typical web application. The problem with Java is that there's simply too much boilerplate and the development flow is too slow. With Go, the language and standard library are just too barebones. Python has some advantages over JavaScript, but not enough to make the smaller ecosystem worth it to migrate. As kind of typical for the Python world, the documentation and resources are also not quite on the expected level. Rust is simply too low level and the important core libraries required for web backends are not mature enough.

Of the alternatives I've explored more closely, I think C# with ASP.NET really comes the closest of hitting home. Modern C# is quite fluid to work with and nowadays it has great async support, too. But I still don't find the increase in boilerplate and the extra effort in setting new projects up etc to be quite worth switching over. Perhaps if I was working on something that had stricter requirements on safety, security and performance, I'd consider ASP.NET.

So, we're left with Node, at least for now.

Over the years, I've tried a ton of frameworks and stacks for Node; Express and lots of its middlewares, SailsJS which is a full-blown MVC framework, ActionHero, and a ton of others. A few years ago I started to mainly use KoaJS, which is a fairly minimal framework that enables you to run a simple HTTP server and plug in middleware. It's a very small project, just a few thousand lines of code, and aims to be minimal and compact. I like that!

I think that it would make sense for companies and individual people to start to try and stick with their choices for longer. Switching frameworks and middleware often simply leads to spreading development effort needlessly much. This means resisting the urge to try out new things, at least when working on longer-term projects. Lots of the libraries and middleware we work with we could just try and improve ourselves when they aren't up to something we'd like to do with them.

What I've tried to build for myself and for the company I work for is a kind of a standard palette of commonly used backend middleware that we try to use for basically every project (of course we can extend the middleware to cover what additional functionality a specific project needs, but the idea is to try to keep the core middleware the same).

For the last few years (and I am very proud of it being whole years and not just months!) the middleware stack has been like so:

  • Koa
  • Koa-router - for setting up HTTP routes
  • Koa-body - for parsing the body
  • Koa-cors - for managing CORS settings
  • Koa-request - for making HTTP requests
  • Koa-websocket - for managing Websocket connections
  • Koa-helmet - for setting some security headers etc.

For databases, I additionally use:

  • Koa-pg - for plain postgres access
  • Sequelize - for a little bit more feature-complete SQL database access and querying
  • Mongoose - ..yeah I still use mongo, particularly in prototyping, simply because it's so easy to set up; but postgres is taking more and more of the burden for me.

Then, when developing, I use nodemon to automatically rerun the project when I modify its source files.

The Node instances themselves are almost always behind nginx. Nginx is a really, really nice web server that is perfect for proxying and serving simple static files. Can't recommend it enough. Specifically, I use it to handle SSL for me. My Node applications do their thing in HTTP and WS, while Nginx handles the outgoing HTTPS and WSS connections. I also use it to combine several services together. E.g. at work our main backend is split to a bunch of services for handling i.e. file upload, user management, log keeping, and our specific projects. Nginx then does the front-facing routing, so you can call e.g. my.com/api/user and my.com/api/bookstore.

For managing processes on the production and staging services, I use pm2. It's another very nice and expandable tool. It can restart a hanging application, it keeps logs (for log rotation, you need a plugin), has monitoring and a sane API to hook into. Fully suitable for large-scale production use.

With this stack, I am now covering several use cases. Firstly, it's relatively straightforward to get into for anyone already familiar with modern JS and Node. Secondly, it uses only tools and libraries that have been around for several years, so we have at least some expectations for maturity. Thirdly, it's still powerful enough to allow for scaling if that is required. For example, nginx can be configured to act as a load balancer, pm2 has a clustering mode and Sequelize supports master-slave read replication for MySQL.

On The Frontend

Moving to the frontend, we find ourselves crossing over from an obnoxious amount of frameworks and middleware to an absolutely completely utterly hellishly ludicrous amount of frameworks. There's Angular, React, Riot, Vue, there's Mustache, Handlebars, EJS, Underscore, there's Semantic UI, Foundation, there's Sass, Less, Stylus ...

Sigh

So for me, what makes sense is to just backpedal, backpedal and backpedal some more and ask, why do we need these frameworks? Here's some points that I can identify; one, managing state and data as it's updated from and to the backend; two, making it easy and quick to add UI widgets; three, maintaining separation of concerns; four, making writing HTML and CSS as easy as it can be.

Some of these concerns are not as acute with modern-day JS, CSS and HTML as they were say 5 years ago. I strongly believe that whenever it's reasonable, we should stick with the basic tooling. When a framework, preprocessor or a language superset no longer is comprehensible with a good understanding of the basic tools, it needs to be thrown out.

Let's start from CSS. Modern CSS already has the widely supported var() function, which allows the use of variables:

:root {
  --main-bg-color: #ff00ff;
}
element {
  background-color: var(--main-bg-color);
}

And currently, there's lots of other features from pre- and postprocessors coming to native CSS, as showcased in this blog post.

Right now, yes, we still need Sass, but once these new features start dropping in and are getting wider support, I say it's time for some farewells.

For many projects, I believe that it's no longer necessary to include any large framework, not even jQuery. Using latest JS, HTML5 and CSS3, any small to small-medium project can just as well be done using standard technology only. The benefit here is that any person coming to work on it should immediately understand the code without needing to first dive into framework specific tutorials or documentation.

When we do need a framework, it's usually down to two reasons; because we have a lot of dynamic data that is often updated to and from the server or because we have a very UI heavy application for which we don't want to write our own widgets.

For the first use case, the one framework I find to be the most intuitive without sacrificing on (too much) performance is Vue. You can easily write Vue without webpack or other compilation tools. It has a fairly good standard way for setting up webpack when you need it. It comes with sane defaults for e.g. CSS preprocessing (offering Sass, Stylus, Less). It feels modern and takes advantage of the bleeding edge features of JS and HTML5. Vue is also pretty modular. You add e.g. data store plugin separately for making it more fluid to handle a larger state.

When I've needed more complex UI widgets, I've found Webix to be pretty nice. Unfortunately, it's not exactly a breeze to integrate it with e.g. Vue apps, so I've found myself not using it too much anymore. Webix has their own web application building thing, called Webix Jet, which is a paid tool. It does look quite good though, so I could even see myself considering its use if there was a suitable project..

For the other UI needs, there's your basic Font Awesome for icons, Vuetify for Vue, and so forth.

To sum it up for the frontend, here's what I try to promote for use in my own projects and at where I work:

If the project is simple:

  • Standard modern HTML5, CSS3, JS

If the project is more complex:

  • Vue w/ Vue's standard webpack setup
  • Vuetify for UI components
  • Font Awesome for icons
  • Sass for CSS preprocessing

Client Side Rendering Only

I'm a strong proponent of always separating the web backend and frontend to their own projects and tools. The backend is "just" an API for the frontend. The reasoning is simply that this acts as a clean separation of concerns and makes handling the mental load easier when you can focus on one thing at a time. Additionally, in the end, I find that this approach actually lowers the amount of development needed. If you're adding server-side rendering, then your backend needs to be both a plain API server while also being tailored to be able to do server-side rendering. Meanwhile, your frontend still needs ways to dynamically talk with your backend.

In my view, any talk about supporting users without JS for your typical web project is also pure nonsense and simply unrealistic. That train passed about 10 years ago.

Summary

So above are the middleware, libraries, frameworks and UI kits I use for web development. In summary, the important thing is not that everyone adopts what I use; the important thing is that I and others spend a little bit of time in finding a stack for themselves that they can use for several years. This increases productivity as you don't need to constantly learn new things for every project.

At where I work as a tech lead - Uplause Ltd - I've mostly adopted the above, though our frontend is not my sole domain so there it's a bit different. Either way, I strongly advocate for companies and individuals to try and stick with their tools.

When our tools are insufficient for us, our first instinct should not be to switch them, but to see if we could improve the tool to do what we need.

TL:DR; Standardize your stack!


comments powered by Disqus