Unexpected UnhandledPromiseRejection in AWS Lambda (Node.js)

We expect AWS Lambda to run just like our local Node.js, but what happens when it doesn’t?

It turns out that if we’re not careful, our assumptions about Lambda can cause some very mysterious bugs

---

Have you ever had these errors pop up in your Lambdas?

An unhandled promise rejection from a previous execution:

A connection reset from seemingly nowhere:

This is some spooky stuff. I had these popping up in my code, and after some investigating around, it turns out they are both caused by the same problem. What could it be? Come take a closer look!

async/await in Node

Let’s begin with a pop quiz: what does this script do?

const sleep = (ms) => new Promise(res => setTimeout(res, ms))

async function x() {
	await sleep(1000);
	console.log('Hello')
}

x() // no await!

1. It quits immediately and prints nothing
2. It does nothing for 1 second and then prints 'Hello'

After doubting yourself for a little, maybe you picked option 2. If you did, then you would be right. But I bet a part of you is still wondering: why doesn’t the script just quit right away? After all, there is no await, and we have reached the end of the script 🤔

This happens because the call to x() even if not await-ed, creates a Promise. This creates a pending callback in the event loop.

The event loop will keep our program running until there are no pending callbacks. When x() finally finishes (and ‘Hello’ was printed), there will be no more pending callbacks, and then we can quit in peace.

Sounds easy enough, doesn’t it? Well…

async/await in Lambda

Now let’s run the same code in AWS Lambda (with a handler function):

const sleep = (ms) => new Promise(res => setTimeout(res, ms))

async function x() {
	await sleep(1000);
	console.log('Hello')
}

exports.handler = async () => {
	x()
}

Run it a couple of times and…

Take a close look. The first execution ends without logging anything. One minute (not one second) later we run the second invocation, and we see the log ‘Hello’.

What is happening here? Well… turns out that the event loop of the first execution bleeds into the second!

This means all pending callbacks from the first execution are ‘paused’ when the first execution ends. When the next execution comes over, it picks up pending callbacks and runs them.

You might already start to imagine the chaos this can cause... here's a little taster:

const sleep = (ms) => new Promise(res => setTimeout(res, ms))

async function x() {
	await sleep(1000);
	await fetch('www.google.com')
}

exports.handler = async () => {
	x()
}

Will get you this:

As if this unexpected behavior wasn’t enough, since we did not await this failing function, we get an UnhandledPromiseRejection in our next execution**, killing an innocent new Lambda**!

We also need to watch out for Promise.race: all promises that didn't win the race will keep running, and this can bleed into the next execution.

But why is this behavior happening? Is there a way to fix it?

The problem

When we use async handlers (exports.handler = async () ...), Lambda finishes the execution when the handler returns (and NOT when the event loop becomes empty). But not all is lost…

We can fix this issue in a number of ways:

• await everything. Our handler will return after all work is finished. I know, very smart…

• Set context.callbackWaitsForEmptyEventLoop = true in the first line of the Lambda. This makes our lambda wait for the event loop to empty to exit.

• Use a non-async handler, which by design provokes the expected behaviour from our first example:

  exports.handler = () => { // No async!
  	x();
  }

In any case, keep in mind that keeping your Lambdas running could increase costs, and that waiting for the event loop to empty is a design choice, and it will depend on the problem you're trying to solve.

I hope this article helped you out if you came across this problem (it drove me nuts for a while for sure!).

See you next time!