Blog

  • Using spies as a way to test side-effects in Node

    You're chugging along writing tests, but then you run into a scenario where you need to test a side-effect. Maybe that's a call to a database, or a call to an external HTTP endpoint, or just making sure a function gets called.

    Maybe you're not used to setting up tests for these scenarios.

    So you do some searching around and figure out you need to use something called "fakes".

    But there are different kinds of fakes - spies, stubs, mocks... which do you use?

    In this post we'll go over one of those types of fakes - spies - how to use them and when you might want to.

  • Using GitHub to determine work-life balance at potential new jobs

    The scenario: you're looking for a new job and have found a few potentially good options, but you have no idea what working there will actually be like...

    Are you going to like your coworkers? Are you going to like the work? Are you going to be working insane hours?

    Those first two questions can be answered by interviewing and talking to people who work at the company, but that latter question - are you going to be working insane hours? - is traditionally more difficult to answer.

  • Simplifying local dev setup with Docker Compose

    If you've ever had to deal with setting up a Node.js project in which you had to install a bunch of things - like MySQL/Postgres, Redis, etc. - and then run some setup scripts just to be able to get the project running locally on your machine...

    ...then you've likely experienced the pain of losing half a day - at least - to solely just getting set up.

  • Using supertest to avoid manually testing your endpoints

    The scenario: In order to test your endpoints/routes in Express/Koa/whatever you're using, you might currently be using a tool like Postman to send HTTP requests and make sure you're getting back the expected responses / the right code is being executed.

    Or maybe you're testing the routes from the front-end of your application.

    Either way, the problems with these methods are: - they're manual, as opposed to automated - they make it difficult to test error scenarios.

  • Setting up your Node package to be run from the command-line

    I've got my Node package built, now how do I run it as a CLI tool?

    Have you ever used Node packages like knex, aws-cli, or mocha and wondered how they are able to be run from the command-line?

    Something like:

    $ my-awesome-tool -v
    

    A lot of people think of Node apps as libraries that you include in your code... or Node REST API's that live on the server. But the ability to build CLI's is sometimes overlooked.

  • Should one Express controller call another?

    When you're working on an Express REST API, you may run into a situation where you need to make a call to fetch some data / do some things from your controller and then take that data and do some more things with it...

    ...and it just so happens you have another controller that returns that data you need / does those same things already.

    So naturally the question arises, should you make that controller to controller call?

    I mean, if it has what you need... why not, right?

  • Awaiting multiple requests to finish using Promise.all

    The scenario: you want to make multiple requests at the same time, and wait for them all to finish before returning all the data. Or, alternatively, you don't need to return any data but instead just need them all to execute before the function returns.

    Maybe you're looking to batch similar requests into X number at a time.

    Or maybe you need to wait for the requests to finish before returning a webpage or response.

  • Express-approved middlewares to use in your API

    When you’re starting up an Express project, it can be confusing to figure out all the modules you need...

    ...and this is no different when it comes to middleware.

    It doesn’t help that – out of the box – Express is "batteries not included."

    Express just isn’t very opinionated. It provides that base and you’re left to piece the rest of the project together.

  • Faking errors to test error scenarios in Express API's

    You've written tests for your Express app.

    You've got most "happy path" test cases covered. Under normal circumstances, your API works as expected.

    But now you need to write a test for how your API handles an error. You want to test that your API returns a HTTP 500 status code, for example, if there is an internal server error.

    The problem is... under normal circumstances your code doesn't encounter an error scenario...

  • Set up your Node project to run tests locally and on CircleCI

    You're creating the backend API for your new Node.js service and it's come time to setup Continuous Integration / CI so you can actually deploy the service.

    CI pipelines can handle a lot of different tasks (building, linting, running tests, checking dependencies, publishing the package if it's a module, etc).

    But here we'll just focus on setting up your project to be able to run tests on CI - using CircleCI as our job runner.

  • How to use `import/export` in Node without Babel

    Have you ever found yourself wanting to ditch using require for your Node imports, ditch writing code like this?

    const knex = require('knex')
    const itemService = require('../services')
    

    If you've been writing any modern client-side JavaScript with React, Vue, etc., you've been importing code like so:

    import React from 'react'
    import TodoComponent from './components'
    

    It would be so great to be able to write in the same style in Node for your server-side code.

  • Awaiting or just returning asynchronous values in JavaScript async/await functions

    When you're writing asynchronous code in JavaScript using async/await functions, it can be confusing to know how to handle returning an asynchronous value. Do you need to await the promise before you return? Or should you just return the promise?

    Let's look at some examples:

    async function getAvailableItems(items) {
      const formattedItems = items.filter(item => !item.name)
    
      try {
        // checks in database
        return await checkIfItemsAreAvailable(formattedItems)
      } catch(e) {
        console.error(e)
      }
    }
    

    How is that different than this?

    async function getAvailableItems(items) {
      const formattedItems = items.filter(item => !item.name)
    
      // checks in database
      return checkIfItemsAreAvailable(formattedItems)
    }
    
  • Possibly the greatest value in having tests for your code is...

    As developers, we constantly hear that we should be writing tests...

    All good developers know how to write tests, and they write them!

    But why?

    We're told this will help prove our code is correct...

    Or that it will prove we've met requirements...

    Or that it will allow us to make changes without worrying if we broke something...

    Or that it serves as a form of documentation...

    And while all of those are true (for the most part at least- your code might not be truly bug-free unless you use something like formal methods/verification), I think possibly the greatest value in having tests for your code is in having documentation of inputs and outputs for your functions.

  • Using functional programming to avoid intermediate variables and nested functions

    Often when we're developing a piece of code, we need to take one starting value and apply several functions to it before we return that value.

    Something like:

    const incompleteTasks = getIncomplete(tasks)
    const withoutBlockedTasks = getNonBlocked(incompleteTasks)
    const sortedByDueDate = sortByDueDate(withoutBlockedTasks)
    const groupedByAssignee = groupByAssignee(sortedByDueDate)
    // etc...
    

    The problem with this is that it's difficult to read. Whenever you add intermediate variables (incompleteTasks, withoutBlockedTasks, etc.), you have to track which ones are passed as arguments to the next functions. So you do a lot of variable tracking when you're reading the code. And why create a bunch of intermediate variables if we don't end up using them anywhere else? It feels like a waste.

  • How to run a npm package from the command line

    Have you ever come across a tutorial with instructions like the following?

    Run npm install knex, then run knex migrate:make migration_name

    That's great that you can run the npm package you just installed (knex, in this case) from the command line, but what's usually left out is how you actually go about doing that.

  • How to run more than one command as part of a npm script

    A common scenario: as part of your npm start script, you need to have more than one command run (like webpack --config webpack.server.js and webpack --config webpack.client.js).

    Up until now you might have only run one command per script - often npm start just does something like node server.js.

    So is there actually a way to run more than one command?

  • Why would you download a Docker image vs. just building from the Dockerfile for local dev?

    When you're working with Docker you'll typically have some sort of image library/registry, like Docker Hub, where built images are uploaded to.

    Then when you need to use that image (let's say you need to spin up a REST API for the UI you're developing), you download that image from the repository and create/start the container.

    Normally downloading those images doesn't take too long, but depending on your connection speed and the size of the image, it could take a while to download it.

    Either way, you might be wondering why you would download it if you could just create the image locally with docker build using the Dockerfile?

  • A better way to sell your skills as a developer than just "hire me!" or mass applying

    There are few things that bring developers more absolute agony than the job search.

    It's weird that even though software development skills are probably the most in-demand of all time, of any career, we still face such difficulties in searching and interviewing for new jobs.

    Applying online, asking friends for referrals, submitting a code test only to wait for weeks to get rejected without a reason at all... the process is one that is quite often filled with frustration, ups-and-downs, and feeling like you have no control whatsoever...

    And that feeling of lack of control usually hurts the most.

  • Understand how to approach designing queues in Node

    A new scenario you might not have faced before: you have a long running task (like saving to a bunch of different databases, or processing video) that takes a while to process and it's currently causing your REST API response times to be way too slow for the end user.

    After some research, you've realized adding a queue to your architecture would solve your problem.

    The only problem is, working out how a queue actually works is confusing.

  • Should you log the Express req object and external API responses?

    Logging as much information as you need to be able to troubleshoot, understand what happened during a session, and even for analytics purposes is something all apps need to have in place before going to production.

    You've likely got some logging in place - things like errors and successful transactions (if you're looking at logs for analytics/metrics). But now you're wondering what else you should be logging so that you have everything you need should you face some issues in production.

    And you might be thinking the Express req object (request object), containing all the information sent to your API, would be a great thing to log. "Great, this will give me all the information about the session!"

  • Executing arrays of async/await JavaScript functions in series vs. concurrently

    When dealing with an array of async/await functions (which return Promises), it can be tricky to figure out how to execute them all in series (one-at-a-time) and how to call them in concurrently (not one-at-a-time, executed during overlapping time periods).

    Maybe you've been trying to execute them in series, but they end up executing out of order. Or maybe you've been trying to execute them concurrently, but they end up executing one-at-a-time, in series.

    In this post we'll explain both methods.

  • Applying the callback -> async/await conversion process to a real-world example

    This is a follow-up to my post on the process for converting callbacks to Promises and to `async/await` functions

    In that post I stuck to using setTimeout as an easy way to introduce some asynchronicity into the code. But I understand that for some people, they need more real-world examples to read and play around with in order to truly get the concept to click.

    So that post was more about the process, and this one is more about implementation. In this post we'll skip promises and go directly from callback to async/await.

  • Real world testing: Using business and technical requirements to know what to test

    This is the next post in my ongoing series on knowing what to test in your Node applications by applying recipes to real-world scenarios.

    In the first one, we went over scenarios you should cover in your tests when calling a database from a Node service.

    And in the second one, we covered what scenarios to write tests for when your applications calls out to an external API.

    This post will be a slight departure from the previous two, but still cover the same ground.

  • How to rewrite a callback function in Promise form and async/await form in JavaScript

    You should really use Promises or async/await here to make this more readable

    How many times have you posted some code snippet when trying to get an answer to your question, and someone ends up pestering you about this? Now, on top of whatever problem you already have with your code, you have another thing you need to learn and "fix"...

    Or what about dealing with refactoring an existing, callback-based codebase at work? How do you convert them to native JavaScript Promises? It would be so great to be able to develop using modern JavaScript and start making use of the async/await functionality...

    If you knew how to avoid callbacks, you could post your code online when asking for help without people asking you to rewrite it and not actually answering your question.

  • Why you should isolate Express from the rest of your Node application

    You might have heard that you should "always layer your app" and "never let logic leak into other layers" before.

    Those are statements made all over your favorite blogs, in "must read" programming books, and at tech meetups and conferences.

    But you might have wondered why exactly it's a problem! So called "best practices" are often presented as such, but without an explanation of what will happen if you don't follow them and how they became best practices in the first place.

    So let's look at one issue you'll run into with your Express REST API if you don't structure your app by layers. And then we'll look at an easy solution for avoiding this problem in the future (and fixing it if you have the issue).

  • Real world testing recipes: Node service that calls an external API

    This is the next post in my series on knowing what to test in your Node applications by applying recipes to real-world scenarios.

    In the first one, we went over scenarios you should cover in your tests when calling a database from a Node service.

    In this post, we'll cover another very common real-world application: a Node service that calls an external REST API/endpoint. "External" means it is an application outside of our own - think the Twitter REST API, Google Maps API, or even an API internal to your company, but not part of your application.

  • Project structure for an Express REST API when there is no "standard way"

    Do any GitHub or Google search for REST API structures using Node + Express and you'll find very few of them follow the same organization.

    What's even worse is, while there are lots of good tutorials out there, many of them have no structure at all. They just put everything into a 200-line server.js file and call it a day...

    One of the best things about Node can also one of the most painful - there are few (if any) true conventions.

    Sure, there are recommend ways of doing things. But it's such a flexible platform that you can often pick any way of doing something and it will likely work.

    But even without conventions, developers want to know the best way to do things. And when it comes to REST API's (and Node projects in general...), everyone seems to feel like they're flying blind when it comes to structuring the project!

  • Avoid manually prepending '/api' to every Express route with this simple method

    Have you ever been frustrated by having to always manually add '/api' to the beginning of each of your Express routes? Usually I see this come in two forms:

    Form 1: In app.js file:

    app.use('/api/users', require('./routes/users'))
    app.use('/api/posts', require('./routes/posts'))
    app.use('/api/comments', require('./routes/comments'))
    app.use('/api/subscriptions', require('./routes/subscriptions'))
    

    Form 2: In routes file:

    router.post('/api/users', users.createUser)
    router.post('/api/posts', blogpost.postBlogpost)
    router.post('/api/comments', comment.postComment)
    router.post('/api/subscriptions', subscription.addSubscription)
    

    Either way- it looks messy, you have to remember to add it each time if you aren't copying/pasting, and it seems like there should be a better way of doing it.

  • Tried TDD and didn't realize the benefits? Try it the next time you get writer's block

    Have you ever tried Test-Driven Development (TDD) thinking it would be the "holy grail" it's often made out to be, only to end up feeling like it was pointless?

    Maybe it didn't add any benefit to your code. Maybe writing your test first, then the code after felt uninspiring or limiting, or just the wrong way to do things, especially since the way programming is taught is code-first, not the other way around. Or maybe it just felt like a chore.

    All the best developers seem to talk about TDD like it's the only way to code, and if you're not doing it that way, you're wrong. So you really want to like it. But if you've tried it and you didn't like it for any of the multitude of possible reasons, what is the point in practicing it at all? If only you could have that "aha moment" that made TDD make sense, you might actually enjoy it and feel like a "real developer."

  • Why should your Node.js application not handle log routing?

    It is not the responsibility of the application to route logs.

    12 Factor says that logs should go to STDOUT. WAT? WHY?

    I just configured my whole application code to write logs to custom log files. What's wrong with that?

    Logging is one of those things that can sometimes be a black box for developers. Maybe you have a dedicated DevOps person who takes care of logging infrastructure for you, or maybe it's your first time working on this side of things.

  • Should you use a logging framework or console.log() in Node?

    The console module is usually the first tool Node.js developers reach for when handling logging in an application. It's easy to use, native to the platform, and easy to read.

    But then you might stumble upon a logging framework like Winston or Bunyan. The basic power of those tools might be attractive enough for you to consider switching.

    But should you? How do you know which one you should use? Should basic console.log’s be avoided entirely and not even used?

  • Explain JavaScript unit testing like I’m five

    Unit testing is so critical to good software development, yet for beginners (and many experienced professionals as well) it's something that can feel alien and uncomfortable at first. It might be something you know you should be doing, but haven't had time to learn, or tried to and didn't get very far. It might also be something you've never even heard of before.

  • Which of the 635000 npm modules do I choose?

    If you've spent any time in the Node or Front End JavaScript world, you know there are hundreds of thousands of modules to choose from.

    Developers constantly ask things and express pain like:

    What's eating away at us is which modules to choose...

    What is the difference between X module and Y module and which one is better?

    npm is great but those modules might be useless in 6 months, a year or later depending on how they're supported.

  • ../../../../ escaping relative path require hell in Node.js

    ../../../../../Having/to/write/code/like/this when you're requiring local modules is a sight for sore eyes. Some consider it one of the bigger pain points with Node due to poor readability when trying to figure out where that module is in the directory structure. Even more painful than reading it is having to think about how many levels you have to walk up then back down when you're the one writing the code (some IDE's like newer version of VS Code help with this but not all do).

    Fortunately there are easy solutions to fix this. Let's take a look at some:

  • Do you need to queue jobs in Node.js?

    If you're coming to Node.js from a more synchronous language/environment such as Ruby on Rails or Java, the concept of asynchronous processing can be confusing. All the literature talks about Node being async, so it's easy to get the understanding that it's all asynchronous. But not every method in Node is async and this is important to be aware of. Likewise, not all Node modules are asynchronous either.

  • Why does async/await in a .forEach not actually await?

    If you're trying to loop over a list while using async/await in Node.js (or the browser, for that matter), reaching for the .forEach array function might seem like a natural choice. Let's say you go that route, fire up your tests or your application, and expect the synchronous-reading magic of async/await to do what it says and actually await the promise.

    What you get instead is either a result in the incorrect order or even the dreaded ReferenceError: 'x' is not defined if your application relies on this return value somewhere else.

  • Using a designated point person in the office to bridge the distributed employee gap

    If you're one of the few remote developers on your team and you're new to the company or working with people you've never worked with before, you may have experienced the pain of relying on people who are unresponsive. This is easier to do when you are remote and they don't have a face to match the name or much of a relationship built up. Instead of familiar faces in the office, they (and you) are just names on a screen. And unlike being in the office you can't just walk over to their desk to get what you need.

  • Questions to anticipate when asking your manager if you can start working remotely

    Deciding that you're going to ask your manager if you can transition from your in-office software developer job to moving to a new city and working 100% remotely can be a big decision. There are likely tons of concerns you have about how to approach this. You've probably practiced your pitch hundreds of times, thought through all kinds of details and prepared for every scenario you could think of.

  • Easy way to make Angular 2 services configurable

    There are several ways to make configurable components and services in Angular 2 / 4 - APP_INITIALIZER, providers, dependency injection, etc.

    But these can sometimes be confusing to use. An easy way to make your services configurable is to simply use a class setter in your service. An example can be found in the library I recently built ng-idle-timeout. (There are a couple libraries that do something similar, but were overkill for the most part)

  • How to use your on-site trips strategically as a remote employee

    In organizations that aren't fully remote, remote employees can sometimes be forgotten about, seen as black sheep or as separate from the group. To your colleagues, you can be seen as that weird cousin people only see for the yearly family gathering and who people sort of know but not really. If you're a remote worker you probably have had that feeling of being excluded from the group, or showing up and people forgetting who you are. Even if you've had plenty of interactions with your colleagues in the past (maybe you worked on-site for a couple years), you're not on people's minds as much.

  • Remote Work As Innovation

    The recent obsession with capital "I" Innovation that has been rapidly gaining momentum the last couple of years has permeated across a majority of large, traditional companies with business models, technologies, and processes seemingly stuck in the past. Fear of competition and thus declining profits and stock prices appears to be the primary motivator behind this obsession, with many of these companies desiring to be more nimble and forward-thinking. This has given rise to the so-called Innovation Group within large organizations, corporate partnerships with startups, and hiring of expensive consultants and consultancies to come in and tell these companies "you should emphatically embrace our way." These actions, often made very publicly as to cast off any prior signifier of "old," seem to be a race to avoid being last to adoption. Through looking to startups (or more nimble companies), these old titans hope to bathe in the fountain of youth and emerge automatically renewed.

  • Calm Technology - CTA train tracker

    This is the first in a series of projects that will be developed around the idea of applying "calm technology" to product design. Coined by Mark Weiser and John Seely Brown at Xerox PARC in 1995, calm technology can de defined at a high level by the following:

  • Setting up Chrome Extensions for use with ES6

    First time setup of Chrome Extensions can be painful if you've never done it before. Add to that setting them up for use with ES6 and you can end up spinning your wheels longer than writing code. I recently went through this while creating Reading List, which makes heavy use of ES6 as well as Ramda for the functional work. While Babel setup is fairly easy, the module loading presented some challenges. Having originally gone with SystemJS I faced a lot of difficulty in getting the tests to run. After switching to Webpack, for all the horror stories I had heard about it, the issues I was facing were resolved within the hour.

  • ES6: Mutability of const

    When first hearing about const in ES6, I was excited about the possibility of having immutability in native JavaScript. For developers programming in the functional style, this would have come in handy, but it turns out const is not actually immutable. It allows mutable properties. For example, all of the below is valid:

  • Functional Programming as the Paradigm for IOT

    As the Internet of Things reaches maturation and begins to become commonplace in our lives, the technology used to support IOT must be chosen well. With potentially millions of devices connected, developing applications to support these devices and the data they produce, while converting that data into something meaningful, will require thoughtful attention to technology choices. In building any system, attention to architecture and technology stacks is important, but if IOT delivers on its promise of scale, the technology implications will be very different than what we've had to solution and develop for before. It will not be enough to simply "use what we've always used" and to keep building things the way we've been building them. The challenges are far too complex to not take a step back and look at other options.

  • PM software should include a ROI feature

    Project management software, and the project management practice in general, should include a Return on Investment feature.

    The problems with modern-day implementations of Agile and its various flavors have already been detailed by many (here's my favorite), but like it or not, the practice is here to stay.  That is, here to stay until the backlash becomes so strong that the same entities who sold its unsuspecting victims on the practice are forced to go back to the drawing board and invent (or rather, vampirically subvert) another methodology that will in turn be sold back to the same companies.

  • ES6: Destructuring

    This is the first post in a series I'll be doing about new ES6 features.  The goal is not to merely explain the concepts, but to also show "real-world" - or real enough - applications of the concepts so that you can understand why and when you might use them.  Hopefully you will be able to start recognizing scenarios or areas in your codebase that could benefit from these new features. After all, new language features should be used to not only help us write cleaner, more expressive code, they should also help us or even challenge us to think about the way we solve problems.

    The first topic feature that will be covered is destructuring.

  • Authoring Yeoman Generators

    The last couple of days I've been playing around with authoring a Yeoman generator for scaffolding out a Sketch app plugin.  While it's not completely done yet, it's in a "good enough/just ship it" state to put the source on github.  I'll be doing some posts in the future on how to create your own Sketch plugins, for which this generator will come in handy, but the purpose of this post is to go over some of the hurdles I faced and some "not-easily-found" documentation for those who are in the process of building their first Yeoman generators.  The existing documentation is pretty helpful but as with any software project, you sometimes need to know where to look to find the information you need.