Why won’t web work

(or the trip hazards of testing from outside a VM)

A problem resolved…

As someone new to web development I ran into an issue the other day that baffled me for an inordinate length of time. As I am essentially required to run Windows as the O/S of choice in my organisation, but my natural habitat is Linux I tend to spend a lot of my time developing my applications and scripts within a virtual machine.

I am a big fan of the Python ecosystem, and naturally, as I test the waters with web development I’d like to investigate pythons tools and frameworks. I also made a change the other day to my process where instead of running a browser internal to my VM, for testing and viewing I decided to run the browser from my host machine.

Running the simple HTTP server in python using python -m http.server was one of the first services I tried. A quick jump onto Chrome later I was initially perplexed that it wasn’t showing up.

However, with not much searching I quickly realised that the default Network setting on VirtualBox (NAT) does not allow connections out unless I explicitly open a port. With that stored in my memory bank, I went on my way, now thinking that I knew how to solve similar issues in the future.

Where did it all go wrong

At this time I had come across the excellent grip package (Grip on GitHub) which allows you to preview markdown files using GitHub’s API. This is a very helpful package for previewing README.md files and more before pushing to a hosted repository. The problem was I couldn’t get it to work on my host machine. I messed around looking for answers for a while without success, and demands on the job meant I just fired up my guests browser and got on with my work. But it bugged me big time.

I then had an occasion where the interactive tooling of ipython notebook (now jupyter notebook) would be useful. I fired up the server, opened the appropriate port, and lo and behold the host machine couldn’t see it. Frustrated, but busy I bit my tongue (swearing’s not great in a shared office) and again worked internally to the guest and got on with it.

Finally I was doing some playing in my own time with Flask, and this time when I came across the same problem I snapped. I binge read everything I could find. Flask specific docs, and then thinking back IPython docs as well. I tried python’s SimpleHTTPServer again (having forgotten I’d successfully used it in the past) and that worked.

The penny drops

OK, I realised, VirtualBox is forwarding ports correctly. They all work locally so there is nothing wrong with each service. I’m just not getting it. Back to basics I fire up a service again and I read the messages printed for a change.

I notice Flask reads: Running on http://127.0.0.1:5000/ and I think good. That’s how it should be, I access it from localhost so it should run on localhost.

I fire up grip, and it’s status message also states: Running on http://localhost:6419/ and I think. OK – what I would expect, so why isn’t it working.

Then I run the SimpleHTTPServer again, and see the message that gets me the tip of the thread. Serving HTTP on 0.0.0.0 port 8000. 0.0.0.0 I think – what’s that? I’ve never seen an IP address like that, so time to research…

Pulling the thread

Now I finally have a clue, a quick search comes up with a fair amount of progress fairly quickly. DavidPostill’s comprehensive answer on superuser.com and the Wikipedia entry quickly give the server specific use of this address as listening on all local IPv4 addresses.

Another rereading of the VirtualBox manual lets me grasp now the implications of VirtualBox’s NAT networking setting (specifically how the service acts as a router between the guest and the outside network).

All that’s left is to tell each individual service to listen on all IP addresses. So in my case we have:

• grip <filename> 0.0.0.0
• ipython notebook --ip=0.0.0.0
• And finally Flask. This requires an edit to the source code, particularly app.run(host='0.0.0.0', ...)

Final thoughts

The biggest thing I have learnt over this whole process is to not ignore those status lines printed to terminal when starting a service. Also, when one things works and another comparable thing doesn’t, look very closely at each point in the chain for important differences.

The most subtle of clues can lead you to increased knowledge and a lot of satisfaction.