Recently I've been busy preparing a project for one of my college subjects. I'm in a team consisting of 5 people, 3 of them being a backend engineer, a web frontend engineer, and an Android engineer, with me in charge of the web frontend. We wanted to build a cross-platform to-do list app that has cross-device syncing functionality. Nothing fancy. As most probably you can tell at this point, the parts of the system consist of an API, a web frontend, and an Android app.
We immediately started planning, creating a Trello board, defining the requirements, creating the design, et cetera et cetera. But then some of us got really busy we ended up off-track with the timeline we initially designed. So now we have to deliver the system in a very short amount of time.
Thus we have to develop the frontend and the backend in parallel. Waiting for the backend to finish to start the frontend development is a big no-no, we wouldn't make it in time. Luckily, one of our members who was in charge of the backend has already created an API documentation previously, albeit the logic hasn't been implemented. So it's just a plain documentation website, the API itself still can't be interacted with.
API Mocking to the Rescue
So, the timeline is short, and the backend isn't ready yet, but we already have an API documentation. This seems like the perfect scenario to try out something new. Which is...
✨ API mocking ✨
I've known the concept of mocking an API to speed up frontend development for quite some time now, which was mainly through this article from Blibli Engineering. But despite that, I've never put it into practice. So now is the perfect time to do it.
For those of you who aren't familiar with API mocking. API mocking is the act of replicating some parts of an API's behavior with dumber implementations just enough to be used by another system. In this case, you can replace the "another system" part with "the web frontend." I use the general term "another system" because this concept is not specific to frontend development, it's just a software engineering concept in general.
For example, the flow of interaction between a client and a simple API typically goes like the following:
But with the mocked version of the API, it can go like this:
As you can see, the mocked version of an API does not have to exactly conform to the actual API's implementation. While in the actual API there are 3 endpoints, in the mock version we can be just fine with 2 endpoints. While the actual API interacts with a database, the mock doesn't have to. But one thing our mock does have to is, it has to have the exact same response structure as the actual API for a given endpoint. For example, if the happy path for the /tasks endpoint in the actual API returns a response with the following structure:
{
"data": [
{
"id": "1234",
"title": "Dummy Task 1",
"status": "Completed"
},
{
"id": "1234567",
"title": "Dummy Task 2",
"status": "In Progress"
}
]
}
Then the mock version of that API has to return a response with the exact same structure for the happy path of its /tasks endpoint. It cannot differ, otherwise, it would be meaningless to create a mock API.
Postman Mock Server
One of the initial solutions that I use to do API mocking was Postman mock server. I was browsing through the internet looking for an easy way to set up API mocking. Then I arrived at the Postman mock server, which seems to provide just that, ease of setup.
To do API mocking using Postman mock server, I just have to create a collection, add examples for each request in the collection that I want to mock, and then create a mock from that collection. You can read more on how to set it up here if you wanted to try it out yourself.
Anyway, my collection looks like this:
The examples are the possible responses the mock API can return. For example, for the sign-in request, I have three examples, each to handle different scenarios, one to handle successful login, one to handle missing password, and one to handle internal server error.
Each of the examples returns a different response, here's the response of the successful login example:
And here's a response of the missing password example:
We can also specify what kind of request an example is expected to handle, by setting the header and body in the request section. For example, here's the request expectation of the successful login example:
This just means that POST requests that go to the /auth/sign-in endpoint that has the same body as provided in the image above should be handled by the successful login example.
And here's the request expectation of the missing password example:
Postman will determine which example to execute based on the HTTP request that was sent to the mock server. There's a separate page that describes how Postman does this. In short, Postman will see the HTTP method, the Postman-specific custom headers, the URL, and the query parameters of the request to determine which example to execute.
You probably noticed that I don't mention the request body being taken into account. But by default, it is indeed ignored by Postman. You need to enable it from the mock server's settings or send a custom header x-mock-match-request-body with value true so that Postman will match the request with the correct example based on the request body.
Even if I did write that, strangely, enabling the request body matching via the mock server settings doesn't yield the desired result for me. I keep getting an error saying no matching requests even though my request's body matches one of the examples' expected request body.
Only by sending the x-mock-match-request-body header then it worked.
I still don't know why though. But if you encounter the same problem, try using the header instead.
Moving Off of Postman
That was pretty easy to set up and it worked just fine for my needs. I can get the development up and running quickly, developing functionalities rapidly, adding features here and there, and moving quickly.
But in the middle of that coding frenzy, I get the following error from Postman:
And I was like:
Yea I forgot that Postman mock server's free tier has monthly limits and I just used up mine in just a few days. Now I have to find another solution to this API mocking stuff.
Mock Service Worker
I have to browse through the internet again. After a little digging, I stumbled upon MSW, which seems to provide what I need. It can run on the browser and node, it seems quite straightforward, it works with React, and I don't have to change the API URL the frontend code is interacting with. So I decided to give it a go.
When running on the browser, Mock Service Worker works by utilizing the service worker to intercept your frontend app's HTTP requests and then return the mock responses. Here's an illustration of it:
When running on node, it works by extending the native http, https, and XMLHttpRequest module. There's no illustration for it though :d
But it's just implementation details. Just know that even if the implementation differs, in general, it still behaves the same way either running on browsers or running on node, and we can use it for use cases in both environments.
Using MSW with Next.js and SWR
I haven't said it anywhere in this article, but I'm building the web frontend using Next.js. Aside from that, I'm using SWR for the data fetching. Luckily, the Next.js community has provided an example of how to use MSW with Next.js, which can be found here.
Looking at the example, it seems like I just need to create a directory to bootstrap the mock worker/server and then create a service worker file in the /public directory. Seems easy enough. Off it goes. I install things, create the mocks directory, set up the handlers, create the service worker file, and wire it all with Next.js, basically just following the example that the Next.js community has provided.
After all is done, I open the browser to see if it's working. Yes, it's working! But, something's not right, the frontend is behaving strangely:
Pardon the lack of styling 😆
As you can see, it detects an error on the initial load, and then after a few seconds, it finally displays the correct data. At first, I thought this is the intended behavior, but then I realize it's couldn't be. I suspect the first HTTP request doesn't get intercepted. Which is confirmed by inspecting the network tab:
Turns out the first HTTP request still hits the real API. Because I don't provide any 404 response for the /tasks in the mock API, so the 404 can only be from the real API.
But why though? What caused MSW to miss only the first HTTP request?
After browsing through the internet once again, searching the issues on MSW, and reexamining the MSW example in the Next.js repo, I still couldn't figure out what did I miss.
I then try running the MSW example provided by Next.js. It worked fine!
Then I try to add some SWR in the code, and voila!
MSW failed to intercept the first HTTP request. Just like what happened on my web app. So now I know the problem has something to do with either SWR or useEffect (I assume SWR most likely uses useEffect, but I never confirmed it).
But why though...?
Well, after a little bit of digging through the issues page of MSW and Next.js, I found the following: https://github.com/vercel/next.js/issues/43284. It's an issue created by the creator of MSW which describes my problem perfectly.
Turns out, the way MSW is set up in the Next.js example caused it to have this bug where it may fail to intercept the initial HTTP request. It's caused by the use of await import which is asynchronous in order to allow MSW to be tree-shaken during build time. But, the function that runs it is never awaited because there is no top-level await, thus creating a race condition where the worker hasn't been up yet but the web page is already on full display.
On that issue, the creator suggests changing from await import to require. Which then I tried. But now I have another error:
Luckily this problem is already described in another issue. This time it's caused by the line that requires the server module is returning a promise for some reason. The creator of MSW himself is not sure why this happens, but he suggests changing from using require to using await import.
But wait... if I do that, then I'm back to square one...
I wouldn't want that, so I try to mix require and await import. require is used when running on the browser, while the await import is used when running on node. Resulting in the following piece of code:
Which surprisingly works! Now MSW never fails to intercept the first HTTP request 🎉
Note:
I don't try to use top-level await to fix the MSW failing to intercept the initial HTTP request problem because I don't want to risk having to waste time dealing with more problems as a result of switching from CommonJS to ES Modules.
If you feel like top-level await can solve it, you can try it yourself.
Conclusion
I've seen many times people still think that the frontend can't be developed unless the backend is ready or at least partially ready, especially among less-experienced programmers such as me and my fellow students. Turns out, by using API mocking, we can make the development of frontend and backend parallel. Which greatly speeds up the development as a whole.
MSW does a nice job as an API mocking tool. With MSW one can freely mimic the real API's behavior according to their needs. It's also quite flexible since we're the one who wires up all the handlers. I also read that MSW can be used for automated testing too, which is interesting, but I haven't tried it.
But I can already see this process of mocking an API is tedious. For an API that has only 3 resources, I've already felt that creating a mock for it is not fun at all. I'm not sure how to improve it though, but I'm sure it could be better. Or, maybe someone out there has already made the solution to it.
Anyway, I think I will continue to use this practice whenever I need to develop a web frontend in a team. It's definitely better than waiting for the backend to finish just to start developing the frontend. I think it's also better than coding the frontend using dummy hard-coded data while waiting for the backend to finish. Because once the time comes to integrate the two, there's gonna be a lot of things to change in the frontend.
If you need to take a look at my codebase and see how I set this up, you can go to https://github.com/checklist-id/web.
See you in the next blog post 👋
