After Washington DC, Brussels is the second most important political capital in the world. All key EU institutions, such as the European Commission, European Parliament and the European Council are headquartered in Brussels. This makes it a home to over 20,000 lobbyists and interest representatives working for over 1,000 industry associations and NGOs. These organisations are also known as EU stakeholders.

These stakeholders are proactive towards political decision makers during legislation cycles, but also serve as a source of information about their respective industries and issues they are representing. They present policy related information in form of position papers, press releases, events, social media activity and presence in traditional media.

The purpose of is to bring and organise all stakeholder activity and information about the key people in each stakeholder organisation, into one place, categorise it, rank it and make it easily searchable and accessible.

The website was to gather information regarding the stakeholders primarily through three sources:

  • their websites (scraping)
  • RSS feeds
  • social media (Facebook and Twitter)

Because it was anticipated that there would be a large number of stakeholders and news sources, one concern was the scalability and robustness of the collection mechanism. All the information had to be collected automatically and displayed in the administration panel for manual approval, modification and publication. We built a mechanism where collection "jobs" were placed in a Beanstalkd work queue to be processed asynchronously by a pool of worker processes waiting at the other end with the whole system monitored by Supervisord.

Another particular requirement had to do with the way user accounts and user logins were to be handled. Users had to be able to log in automatically from specific ip blocks and email addresses that belong to specific domains. These ip blocks and domains would be assigned when an organisation bought a subscription to the service with the purpose being that employees of these organisations wouldn't have to individually set up accounts to use the service all the while retaining the option to set up customised accounts for themselves if they wished.

Custom user accounts have the option to bookmark keywords, categories and stakeholders and also receive weekly email notifications about news items that are added to their bookmarked areas of interest, thus making it easy for them to keep up-to-date regarding trends and developments.