Rome in transit (GTFS)

ivandorte · February 10, 2023, 9:59pm

Hi there!
I share the source code of a Panel dashboard that I built few weeks ago:

Description

This dashboard displays the real-time position of any bus, tram, or train (ATAC and Roma TPL operators) within the Metropolitan City of Rome Capital. The “stream layer” updates dynamically via a Pipe stream every 10 seconds.

Deployed on

GitHub Pages: https://ivandorte.github.io/Rome-in-transit/gtfs-rt-rome/app.html;
Sharing gallery: Not yet (why? the process to convert the Panel dashboard to a Pyodide-based application returns an error due to the presence JS XMLHttpRequest library);

What I’ve done here

Loaded custom Python code from GitHub;

github.com

ivandorte/Rome-in-transit/blob/890b54f0834995e6c024f5dfdc20993c9cde8e2d/docs/gtfs-rt-rome/app.js#L40-L53

      
        
            
            
// Load custom Python modules
            const custom_modules = ['https://raw.githubusercontent.com/ivandorte/Rome-in-transit/main/modules_pyodide/constants.py', 'https://raw.githubusercontent.com/ivandorte/Rome-in-transit/main/modules_pyodide/rome_gtfs_rt.py']
            for (const module of custom_modules) {
              let module_name;
              module_name = module.split('/').slice(-1)[0]
              await pyodide.runPythonAsync(`
                  from pyodide.http import pyfetch
                  module_pyodide = await pyfetch('${module}')
                  with open('${module_name}', 'wb') as f:
                      f.write(await module_pyodide.bytes())
                `);
            }

Loaded protobuf and gtfs-realtime-bindings wheels (not available on pyodide) from a CDN URL;
Used corsproxy.io to bypass CORS errors on HTTP requests;
Used XMLHttpRequest (js) to make HTTP requests of binary data;

github.com

ivandorte/Rome-in-transit/blob/890b54f0834995e6c024f5dfdc20993c9cde8e2d/modules_pyodide/rome_gtfs_rt.py#L58-L76

      
        
            
            
def read_feed_xhr():
                """
                HTTP request used to retrieve binary data from
                Roma mobilità GTFS-RT feed (XHR).
                """
            
            
    url = build_url()
            
            
    xhr = XMLHttpRequest.new()
                xhr.open("GET", url, False)
                xhr.overrideMimeType("text/plain; charset=x-user-defined")
                xhr.responseIsBinary = True
                xhr.send(None)
            
            
    response = xhr.response
            
            
    return bytes(ord(byte) & 0xFF for byte in response)

Marc · February 11, 2023, 4:14am

Hi @ivandorte

Thanks for sharing. It looks like an amazing piece of work.

On my end the requests though cannot get through due to CORS.

Thus I do not see any data

One solutions could be to ask romanmobilita.it to enable using their data from applications like yours by adding the missing CORS headers. I just did asked something similar for KLIFS - the structural kinase database and they just did it.

I hope you get it working. Really learned a lot from looking at your code and how you import extra python files and also custom built wheels.

ivandorte · February 11, 2023, 12:51pm

Hi @Marc,
thanks for the feedback . I sent a request to Roma Mobilità (social networks, forums, etc…) to enable CORS headers on their resources. In case of negative response, I will change the GTFS feed URL.

carl · February 11, 2023, 1:17pm

I really like this @ivandorte great work and one day I hope my busses will open up their live positioning data too

maximlt · February 11, 2023, 6:17pm

The app is working for me:

Cool stuff

Marc · February 11, 2023, 7:42pm

The above CORS issue is on windows with Edge.

I then tried with Firefox. But there is another issue.

Maybe the CORS issue is related to some internal security in my organisation. If I try to open the link https://corsproxy.io/?https://romamobilita.it/sites/default/files/rome_rtgtfs_vehicle_positions_feed.pb?cacheBust=20:45:39 in the error above I see

maximlt · February 13, 2023, 7:41am

Yes probably. The app runs fine for me on my default Firefox browser.