Urban Governance survey website

Survey visualizations and source data

The Urban Governance survey visualizations are generated through a web application which runs in a user’s browser when they visit the Urban Governance survey website. The application uses data from four different files, which are hosted on the website itself and are loaded transparenty by the application:

World map (GeoJSON file)
Used to draw the world map on the “Survey results” page.
Survey results (CSV file)
Contains the actual answers of the survey as given via SurveyMonkey and successively cleaned up.
Titles data (JSON file)
Contains the full text of survey questions and answer labels, in the three languages of the web application (English, Spanish, French).
Short question data (CSV file)
Contains the labels of survey themes and short version of the survey questions as shown on the “Survey results” page, in the three languages of the web application

Updating survey data

Whenever new survey data is prepared (e.g. when new cities submit their responses), one or more of the files referenced above will need to be updated on the live survey website: ypically the Survey results CSV file, but minor amendments may occasionally be required to other files, as needed.

Whereas the data files at the time of the latest iteration of development (May 2016) are included in the WordPress theme, new data files can be managed by site editors by uploading updated files to the WordPress media library (https://urbangovernance.net/wp/wp-admin/upload.php).

Once updated data files have been uploaded, the web application will have to be configured to use these instead of the ones currently in use. Simply uploading new files to the WordPress media library makes them available for use, but they are not actually used until the web application is reconfigured.

It is advisable to test that any updated files work correctly with the web application before configuring this to use them: in order to do so, each set of data files (the four data files listed in the previous section) can be grouped into a ‘Data package’, and any number of such data packages can be defined within the WordPress dashboard. Whereas the ‘live’ web application will use a specific data package for the visualizations shown to users, editors can test alternative data packages by using a special version of the web address used for data visualization pages, as outlined below.

Creating data packages

Firstly, a data package can be created via the Data packages -> Add new menu of the WordPress dashboard.

A descriptive title should be added in the text area at the top of the page where a new data package is configured. This is not displayed anywhere within the Urban Governance website and is only meant for editors to be able to quickly identify data packages. The Permalink field, if left blank, will be automatically filled by WordPress with a label derived from the title given to the data package; however, it is advisable to explicitly set this to a short identifier (using alphanumeric characters, ‘-‘ and ‘_’ characters only: spaces and other characters will be automatically replaced with a ‘-‘ character) as this is then needed when testing alternative datasets.

The next four fields (World map, Survey results data, Titles data, Short question data) need to be filled in by selecting a suitable file. This can either be an older file already present in the WordPress media library (e.g. the GeoJSON file with a world map, which most likely won’t change), or a new file uploaded from the local computer.

All the four fields above need to be linked to a data file, even if this is the same as in the data package currently being used for the ‘live’ web application.

The Notes field can be used for any editor annotations related to the data package (e.g. to list checks or updates needed before making a data package ‘live’). The contents of this field are not used anywhere in the publicly browsable web application.

The final field (Author) can be ignored: it is automatically set to the WordPress username of the editor who created the new data package in the first place.

Testing data packages

Once a new data package has been configured and saved (Save data package button at the top right of the page), it can be tested without disrupting the browsing experience of other visitors: to do so, navigate to the page where the updated dataset needs to be tested, wait for the page to load, then add the following string at the end of the address of the page in the browser’s address bar (but please see note below about city profile pages):

?dp_id=<permalink_of_data_package>

Where <permalink of data_package> is the permalink (as set above) of the data package being tested.

For example, to test a new data package whose permalink was set to 2016v1-test on the “Survey results” page, English version, the address to use is:

https://urbangovernance.net/en/?dp_id=2016v1-test

City profile pages (individual cities) work slightly differently; the string to be added at the end of the browser’s address bar for these is:

&dp_id=<permalink_of_data_package>

For example, to test a new data package whose permalink was set to 2016v1-test on the profile page for Lisbon, English version, the address to use is:

https://urbangovernance.net/en/cities-profiles/city-profile/?city=4240579359&dp_id=2016v1-test

The permalink of the data package is not displayed anywhere on the pages displayed in visitors’ browsers, although it is visible to anyone wishing to inspect the source code of the web application through the browser’s “developer tools”: it is therefore advisable to choose permalinks that don’t contain references to internal processes, authors or reviewers of the data, etc.

If the data package being tested does not display correctly, it is possible to collect information about any issue by opening the browser’s “developer tools” panel (Shift+Ctrl+I on Windows or Linux, Cmd+Opt+I on macOS) and consulting its “Console” tab. A frontend developer or data scientist will be able to further diagnose any issues that may be occurring with a new data package.

Using a new data package on the live web application

Once extensive testing of a new data package has been conducted and no issues are occurring after any adjustments needed, in order to reconfigure the live web application to use the new dataset, navigate to the Settings -> Visualization settings menu of the WordPress dashboard, select the dataset that should be used, then activate the new settings by pressing the Save Changes button. Due to server-side caching, changes may take 30 minutes to 3 hours to be visible to all users.

Offline version of the website

The Urban Governance website is set up so that it is automatically installed as an offline app in recent browsers.

Chrome and Chromium version 51 or later, Firefox version version 48 or later on Windows, macOS or GNU/Linux can install the site for offline use. Safari, Internet Explorer and Edge browsers don’t currently include the features needed to allow offline use of websites.

Although the website is installed automatically in supported browsers without any need for user action, the process may take up to a couple of minutes, depending on the speed of the Internet connection used.

Chrome and Chromium browsers allow to easily verify that the website has been successfully installed for offline use (e.g. if it is necessary to showcase the website on a computer without an active Internet connection). In order to do so:

  • open a new browser tab in Chrome or Chromium
  • press the Ctrl+Shift+I (on Windows or GNU/Linux systems) or Cmd+Opt+I (on macOS systems) key combination to open the browser’s Developer Tools
  • select the Application tab from the top of the Developer Tools part of the window, then the Service Workers menu item from the left-hand side navigation area of the Developer Tools
  • visit https://urbangovernance.net from the active browser tab
  • some information will be displayed about the website’s ‘service worker’ being installed; the status line will initially display a white dot and a message confirming that the “service worker” is being installed; once the dot turns green and the status message changes to #<some number> activated and is running, the website is fully installed for offline use

Updating the offline version of the website

The list of resources (pages, images, visualization scripts, visualization data) saved in the browser for offline use is compiled manually within WordPress: although new versions of pages and images are updated automatically when an Internet connection is available, any of the following require a manual update of the WordPress theme’s code:

  • adding new pages
  • adding new images
  • updating the visualization data or titles/labels
  • changing the web address of a page
  • updating the filename of an image

If any of the above actions is performed, a developer will need to update the list of resources to be installed for offline use and upload the updated WordPress theme to the web server.