Troubleshoot HTTP API requests with mitmproxy

Sometimes you connect a new tool to one of your servers and it doesn’t work as expected. You are sure you follow the documentation or tutorials but you don’t get the expected results. Before you throw away everything, you should check what’s actually going on between the 2 applications. And if none of them supports logging requests and responses, you can use mitmproxy for troubleshooting.

As the name imply (MITM = Man In the Middle), mitmproxy sits between both applications and intercepts all the traffic. You can use it to log the traffic but also modify the content of the requests and/or responses? On the fly. I will not cover that use-case here.

Just point mitmproxy to your API server, and point your new tool to mitmproxy, and witness exactly what’s going between both apps.

Suppose your API server listens on port 1234 and you want mitmproxy to listen on port 5678, then you can use mitmdump --mode reverse:http://apiserver:1234@5678 --verbose --anticache --anticomp --flow-detail 4 for example.

You can even use it in a docker compose stack. Let’s say service frontend talks to service apiserver in the stack. Add another service mitmproxy, point frontend to mitmproxy instead of apiserver, and point mitmproxy to apiserver as described above.

From this:

version: "3"

services:
  apiserver:
    image: local/apiserver:latest

  frontend:
    image: local/frontend:latest
    ports:
      - 80:80
    environment:
      BACKEND=http://apiserver:1234

To this:

version: "3"

services:
  apiserver:
    image: local/apiserver:latest

  mitmproxy:
    image: mitmproxy/mitmproxy
      command: mitmdump --mode reverse:http://apiserver:1234@5678 --verbose --anticache --anticomp --flow-detail 4

  frontend:
    image: local/frontend:latest
    ports:
      - 80:80
    environment:
      BACKEND=http://mitmproxy:5678

If you want a webinterface, then use mitmweb --web-host 0.0.0.0 --web-port 8080 --mode reverse:http://apiserver:1234@5678 --verbose --anticache --anticomp .

Of course you can use it in the the docker compose stack above. Remember to either expose the port 8080 or to use Traefik to proxy it. Note that if you use Traefik, you must rewrite the Host and Origin headers. See this issue on Github. During my tests, I had to rewrite the Host header to 0.0.0.0 and unset the Origin headers.

- "traefik.http.routers.apiserver-mitm.middlewares=apiserver-mitm-headers@docker"
- "traefik.http.middlewares.apiserver-mitm-headers.headers.customrequestheaders.Host=0.0.0.0"
- "traefik.http.middlewares.apiserver-mitm-headers.headers.customrequestheaders.Origin="

Make sure to secure the endpoint, you don’t want everyone to spy on you, do you? ;-)