Building the European Social Innovation Database with Natural Language Processing and Machine Learning

Abdullah Gök, Roseline Antai, Nikola Milošević, Wesam Al-Nabki

Research output: Contribution to journalArticlepeer-review


Social innovation is widely defined as technological and non-technological new products, services or models that simultaneously meet social needs and create new social relationships or collaborations. Despite a significant interest in the concept, the lack of reliable and comprehensive data is a barrier for social science research. We created the European Social Innovation Database (ESID) to address this gap. ESID is based on the idea of large-scale collection of unstructured web site text to classify and characterise social innovation projects from around the world. We use advanced machine learning techniques to extract features such as social innovation dimensions, project locations, summaries, and topics, among others. Our models perform as high as 0.90 F1. ESID currently includes 11,468 projects from 159 countries. ESID data is available freely and also presented in a web-based app. Our future workplan includes expansion (i.e., increasing the number of projects), extension (i.e., adding new variables) and dynamic retrieval (i.e., retrieving and extracting information in regular intervals).

Original languageEnglish
JournalScientific Data
Issue number1
Publication statusPublished - 12 Nov 2022


  • social innovation
  • natural language processing
  • machine learning
  • deep learning
  • text mining
  • named entity recognition
  • Classification

Research Beacons, Institutes and Platforms

  • Manchester Institute of Innovation Research


Dive into the research topics of 'Building the European Social Innovation Database with Natural Language Processing and Machine Learning'. Together they form a unique fingerprint.

    Nenadic, G. & Shapira, P.


    Project: Research

Cite this