If a Node.js library offers synchronous methods that are not just wrappers for the few synchronous methods offered in the standard library, it probably uses deasync or relies on something that does. At a glance, it sounds like exactly the thing that every callback weary Node.js developer is looking for. A way to make an asynchronous function behave synchronously.
1. The original author, current maintainer, and the maintainers of Node.js recommend against using it.
Deasync might sound like the perfect solution, however, even the current maintainers of deasync recommend that you don’t use it if there is an alternative, which there is in every case. Not only that, the maintainers of Node.js have deemed deasync unsafe[ref]https://github.com/nodejs/node/pull/2830#issuecomment-207490532[/ref] and the original author states in the readme for the library that “[deasync is] just a hack and I’d strongly recommend not to use it for anything.”[ref]https://github.com/vkurchatkin/deasync[/ref] Unbelievably, deasync is currently at a 140,000 downloads a week and climbing.
2. Its behavior is not stable and not supported.
Deasync blocks the event loop by calling the Node.js event loop at the JavaScript layer (inside the main event loop). This is done by exposing functionality from libuv and v8 to the consumer. Both modules are non-ABI stable, meaning their behavior could change at any point. This is exactly what happened in 2015, when a user reported that using deasync caused their app to hang in an asynchronous context.[ref]https://github.com/abbr/deasync/issues/21[/ref] Many other users chimed in to say they were having similar problems.
This prompted the maintainer of deasync to create an issue and pull request against Node.js. The maintainers were quick to point out that what deasync does is not supported.
Oop. I just realized you’re running the event loop within the same event loop. Yeah, that’s definitely not supported by libuv. Sorry.[ref]https://github.com/nodejs/node/pull/2830#issuecomment-147784791[/ref]
There is no way to reproduce it using public APIs, so it’s not bug. What deasync does is not supported[ref]https://github.com/nodejs/node/pull/2830#issuecomment-207483888[/ref]
Later on, the maintainers of Node.js declared that deasync is unsafe in addition to being unsupported and the pull request was closed without being merged.
Using deasync is quite literally UNSAFE.[ref]https://github.com/nodejs/node/pull/2830#issuecomment-207490532[/ref]
3. It can cause your app to hang indefinitely.
It possible for “deasynced” functions to hang indefinitely, stalling the app. This is particularly noticeable when a deasynced function is called after trapping process.exit(). Instead of exiting, the program will hang and will not accept additional sigint/sighup signals, requiring the user to find and kill the process with more drastic measures.[ref]https://github.com/abbr/deasync/issues/21[/ref]
One of my first ever projects as a software engineer was writing a module that used etcd v3 as a persistent state store for hubot’s brain. This module was based on @meaballhat’s (Dan Buch) hubot-etcd-brain. The idea was to tap in to hubot’s event api and listen for save events emitted by hubot’s brain and transfer the state of hubot’s brain to etcd.
Hubot emits a save event periodically, and most state management libraries for hubot rely on this save event to synchronize the state of the brain to the persistent state store. The problem with this is that if the state of the brain isn’t changing, hubot is making needless network requests. To make matters worse, if the save events aren’t frequent enough, there is a risk that data might be lost if the state of the brain changes and app crashes before the next save interval.
I wanted to make sure the state store would attempt to save the state of the brain if hubot was told to shutdown. I tried to catch exit signals, however, hubot was doing the same. When hubot caught an exit signal, it called process.exit().
There is no way to prevent the exiting of the event loop at this point, and once all
'exit'listeners have finished running the Node.js process will terminate.[ref]https://nodejs.org/api/process.html#process_event_exit[/ref]
In other words, if there are any asynchronous operations that started before process.exit() is called will not have a chance to complete. I was convinced that I needed a synchronous way to write the state of the brain to etcd when process.exit() is called. So, I added deasync and wrapped my write method with it, and called that function on process.exit(). Imagine something like the code below.
const deasync = require('deasync')
const fs = require('fs')
const net = require('net')
const syncWrite = deasync(fs.writeFile)
const superImportantExitSave = (data) => {
syncWrite(__dirname + '/out.log', data)
console.log('Wrote data: ' + data) // Big fat lie
}
process.on('SIGINT', () => {
process.exit(0)
})
process.on('exit', () => {
superImportantExitSave('Aw shucks!')
})
const server = net.createServer((c) => {
// 'connection' listener
console.log('client connected')
c.on('end', () => {
console.log('client disconnected')
})
c.write('hello\r\n');
c.pipe(c)
})
server.on('error', (err) => {
throw err
})
server.listen(8124, () => {
console.log('server bound')
})
Much to my surprise, my deasynced function failed to write to the file in this case. Even more perplexing, the console.log() that is called after it worked and printed the data to the command line. If I couldn’t trap 'exit', maybe I could trap the same signal that prompted hubot to call process.exit(), 'SIGINT'. Caution, the code below will not exit with a 'SIGINT', for example, when you press ctrl+c. You will need to take more drastic measures like using the kill command to kill the process.
const deasync = require('deasync')
const fs = require('fs')
const net = require('net')
const syncWrite = deasync(fs.writeFile)
const superImportantExitSave = (data) => {
syncWrite(__dirname + '/out.log', data)
console.log('Wrote data: ' + data) // Never gets called
}
process.on('SIGINT', () => {
superImportantExitSave('Aw shucks!')
})
const server = net.createServer((c) => {
// 'connection' listener
console.log('client connected')
c.on('end', () => {
console.log('client disconnected')
})
c.write('hello\r\n');
c.pipe(c)
})
server.on('error', (err) => {
throw err
})
server.listen(8124, () => {
console.log('server bound')
})
Although the process hung up, the data was written to the file, but the console.log() statement was never called. On the surface, deasync sounds like a good solution to edge cases like this one. Unfortunately, the behavior of deasync has unintended and unfixable[ref]As far as the maintainers of Node.js are concered.[/ref] consequences. For example, if you were using deasync to ensure graceful termination, you would actually be forced to ungracefully terminate the app in situations like the one I described above.
4. It could cost you a significant refactor.
It seems that many are using deasync at all layers of their app instead of just the exit layer. When I encountered problems with deasync, I researched the open issues for deasync and found that many users state that if they were forced to use an alternative to deasync, this would cause significant refactors, presumably because data at the top layer of their app comes from a “deasynced” function, or these functions are littered throughout the app.
This is a huge problem for us because changing it to async causes a cascade of refactors, which will ultimately cause nearly all of our sync functions to become async.[ref]https://github.com/nodejs/node/pull/2830#issuecomment-147527492[/ref]
Of course many users reporting issues are maintainers of projects they inherited from others that were created at a time when the problems with deasync weren’t known. However, the problems have now been known for years now. Still, deasync sees 140,000 downloads a week.
5. There are alternatives.
Before you consider deasync, consider the alternatives. In my case, the use of process.exit was the source of my woes. However, it is the hubot library and not my own extensions that call process.exit. It would be better for hubot to allow the consumer to decide when to call process.exit rather than making the decision for them.
In the case of hubot dilemma, I could use one of the few synchronous methods available in the core library to store the state of hubot’s brain. For example, I could use fs.writeFileSync to store the state to a json file. The next time the app loads, I could tell it to read the state from the file. Alternatively, I could use child_process.execSync to call an arbitrary shell command. This could create security vulnerabilities arbitrary input is also accepted and it could also limit the portability of the app between operating systems. To minimize portability issues, you could call your app from within itself via like the minimal example below shows.
const {execSync} = require('child_process')
const https = require('https')
const req = https.request(new URL('https://joecreager.com'), (res) => {
let data = ''
res.on('data', (chunk) => {
data += chunk
})
res.on('end', () => {
// calling process.sdout.write because console.log adds a \n character
process.stdout.write(data.length.toString())
})
})
// parse command line args
if (process.argv[2] === 'exit-save') {
req.end()
} else {
// default if there are no additional command line args
process.on('exit', () => {
// call this script with an additional arg to trigger the the exit-save request
console.log(execSync('node index.js exit-save').toString())
})
process.exit(0)
}
In this example, if I run node index.js from the command line which binds an exit listener to process and then immediately calls process.exit. The exit listener uses execSync to call the same script with an additional exit-save argument. This makes an asynchronous http request and then writes the length of the response to stdout. The response length is then written to stdout via console.log within the exit listener. There are a number of limitations because data passed to stdout must be a string and data returned from execSync is also string (in the form of a buffer). If you were to use this method to turn asynchronous functions into synchronous ones, you wouldn’t have access to JavaScript data structures, although you could JSON encode and decode the data, or do some other form of string conversion, and then use eval to call any encoded functions.
let func = () => {
console.log('Hello world!')
}
const obj = {func: func.toString()}
const json = JSON.stringify(obj)
const newFunc = eval(JSON.parse(json).func)
newFunc()
// Hello world!
All of this is obviously a hack and potentially could introduce security vulnerabilities if input is not sanitized, but it works, and is supported behavior as far as I am aware. Still, the best alternative is to stick to using asynchronous control flow rather than devising hacks to introduce synchronous behavior to asynchronous functions in Node.js. However, if you must do so, know that there are alternatives that don’t involve manipulating the runtime environment with native extensions.
I believe that there is place for synchronous methods in all I/O APIs for Node.js as there are clearly use cases. The missing link is mainly network based I/O APIs, which offer no synchronous methods. These could be useful for command line utilities where concurrency is not a significant issue, and users would be free to choose asynchronous methods when they needed to execute tasks concurrently. Using synchronous network methods would clearly be a problem for applications like servers where many users will connect concurrently, but using synchronous file system and shell commands also problematic in this context. If there are no technical limitations, the omission seems arbitrary.
Bonus: It will cost you time.
When I tried to use deasync I found that trying to get it to work used up a lot of time. I had to figure out what it was causing my app to hang on exit. After I figured what was happening I had to convince other members of my team that it wasn’t going to work and that we needed to abandon the library that used it. My motivation for writing this article was to spare people from losing time I had.
I am not the only one who has lost time to deasync. The people who maintain webdriverio ran into a different set of problems after trying to incorporate deasync into webdriverio to offer a synchronous API for their library.
While everything runs fine on Node v10 it seems to not work for v12. While
deasyncgenerally works in that version, it doesn’t work in the way we run nested promises. So far there doesn’t seem to be an easy fix for it. I also experienced errors likeasync hook stack has become corrupted (actual: 209, expected: 210)[ref]https://github.com/webdriverio/webdriverio/pull/4886#issuecomment-574116222[/ref]
As of June 2020, some folks using a jsx plugin have run into trouble as well. In the end the webdriverio team had to give up on using deasync in their library which definitely cost them time. I imagine that these are not the only examples were deasync has not worked out while burning up a lot of time. Consider reaching out to me if you have an experience you can share.