Shared Tasks

Shared Tasks

We are happy to announce that we will be hosting eight shared tasks:

Shared Task 1: Arabic Financial NLP (AraFinNLP)

• Description: In the AraFinNLP shared task, we propose two subtasks for advancing Financial Arabic NLP in the banking domain: Multi-dialect Intent Detection (Subtask-1) and Cross-dialectical Translation with Intent Preservation (Subtask-2). These tasks focus on accurately interpreting and managing diverse banking data in Arabic-speaking regions. They enhance customer service, automate query handling, and promote inclusivity and efficiency by detecting intent in financial communications across various Arabic dialects. Dialectical Translation ensures NLP models work effectively with diverse Arabic dialects, opening up new applications in customer support, financial news analysis, and improving accessibility for Arabic-speaking populations, making financial services more inclusive and efficient.

• Organizers: Mo El-Haj (Lancaster University), Houda Bouamor (Carnegie Mellon University Qatar), Saad Ezzini (Lancaster University), Ismail Berrada (Mohammed VI Polytechnic University), Sanad Malaysha (Birzeit University), Mohammed Khalilia (Birzeit University), Mustafa Jarrar (Birzeit University), Sultan Almujaiwel (King Saud University).

• For more information, please visit the shared task website: https://sina.birzeit.edu/arbanking77/arafinnlp/ 

Shared Task 2: FIGNEWS 2024: Shared Task on News Media Narratives of the Israel War on Gaza

• Description: This shared task seeks to unravel the layers of bias, propaganda and double standards within news articles in multiple languages, fostering a collaborative exploration of media narratives surrounding one of the most critical moments in recent history. The overarching objective is to establish a shared corpus for comprehensive annotation across various layers, crafting annotation guidelines shaped by the diverse range of conflicting discourses around this sensitive topic.

• Organizers: Wajdi Zaghouani, (Hamad Bin Khalifa University), Mustafa Jarrar (Birzeit University), Nizar Habash (New York University Abu Dhabi), Houda Bouamor (Carnegie Mellon University), Imed Zitouni (Google), Mona Diab (Carnegie Mellon University), Samhaa R. El-Beltagy (Newgiza University).

• For more information, please visit the shared task’s website: https://sites.google.com/view/fignews/home 

Shared Task 3: ArAIEval: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content

• Description: The spread of propagandistic content through mainstream and social media platforms poses a significant challenge, misleading the general public. To counteract this, it is important to automatically identify such content as it supports fact-checkers, journalists, and the public in scrutinizing the information. Building on the interest of the previous year's success, this year ArAIEval (https://araieval.gitlab.io/)  introduces two groundbreaking tasks aimed at combating propaganda in Arabic media. The first task involves the detection of propagandistic textual spans and the identification of persuasion techniques in Arabic news paragraphs and tweets. The second task focuses on differentiating between propagandistic and non-propagandistic memes, utilizing a unique dataset of Arabic memes. These initiatives mark the first efforts of their kind, targeting the novel and complex challenge of propaganda detection in Arabic content.

• Organizers: Firoj Alam (Qatar Computing Research Institute), Maram Hasanain (Qatar Computing Research Institute), Reem Suwaileh (HBKU), Md. Arid Hasan (University of New Brunswick), Fatema Ahmed (Qatar Computing Research Institute), Rafi Ul Biswas (HBKU), Wajdi Zaghouani (HBKU).

• For more information, please visit the shared task’s website: https://araieval.gitlab.io/ 

Shared Task 4: StanceEval2024: Arabic Stance Evaluation Shared Task

• Description: Stance detection aims to identify the position or perspective of a writer towards a specific topic or entity by analyzing his written text. The goal of this shared task is to propose models for detecting writers' stance (Favor, Against, or None) towards three topics (COVID-19 vaccine, digital transformation, and women empowerment). Therefore, we invite participants to submit their solutions for the stance detection task. These solutions could exploit machine learning techniques for stance detection.  Other approaches could also be possible. Participants can approach the stance detection task through single-task or MTL. Single-task learning-based models depend only on the stance data for model development and training. MTL-based models can use other information, such as the sentiment and sarcasm of each tweet, to boost the performance of the stance detection system. The shared dataset will contain the stance, sentiment, and sarcasm information of each tweet. The submitted systems will be evaluated per topic (COVID19 vaccine, digital transformation, and women empowerment).

• Organizers: Nora Alturayeif (Imam Abdulrahman Bin Faisal University), Hamzah Luqman (King Fahd University of Petroleum and Minerals), Zaid Alyafeai (King Fahd University of Petroleum and Minerals), Asma Yamani (King Fahd University of Petroleum and Minerals).

• For more information, please visit the shared task’s website: https://sites.google.com/view/stanceeval/home 

Shared Task 5: WojoodNER 2024:  The 2nd Arabic Named Entity Recognition Shared Task

• • Description: WojoodNER 2024 consists of three subtasks. The first and the second subtasks are "Flat Fine-Grained NER" and "Nested Fine-Grained NER". Participants will be provided with a new fine-grained NER dataset (Wojood+subtypes), which contains 550K tokens, 75K entity mentions covering the parent types, and 47K subtype entity mentions.  The third subtask is "Open-Track NER - Gaza War". While the first and the second subtasks are "closed-track" and thus external datasets are not allowed; in the third subtask, participants are allowed to use any external datasets and tools. This subtask focuses on the Israeli War on Gaza, and participants will be given development and test sets in five different news domains. These subtasks will inspire various methodologies, including transformer architectures and LLMs, to address the nuanced demands of Arabic NLP research.

  • Organizers: Mustafa Jarrar (Birzeit University), Muhammad Abdul-Mageed (The University of British Columbia and MBZUAI), Mohammed Khalilia (Birzeit University), Bashar Talafha (The University of British Columbia), AbdelRahim Elmadany (The University of British Columbia), Nagham Hamad (Birzeit University)

  • For more information, please visit the shared task website: https://dlnlp.ai/st/wojood/ 

Shared Task 6: ArabicNLU Shared-Task: Arabic Natural Language Understanding

• Description: Natural language understanding (NLU) is a core aspect of natural language processing (NLP), facilitating semantics-based human-machine interactions. One of the key challenges in Arabic is ambiguity, that is because Arabic exhibits morphological richness, encompassing a complex interplay of roots, stems, and affixes, rendering words susceptible to multiple interpretations based on their morphology. Ambiguity in language can lead to misunderstandings, incorrect interpretations, and errors in NLP applications. A core NLU task is Word Sense Disambiguation (WSD), and its special case Location Mention Disambiguation (LMD). WSD aims to determine the correct sense of ambiguous words in context, while LMD focuses on disambiguating location mention that is referred to with multiple toponyms, i.e., particular place or location. Both tasks are vital in NLP and information retrieval, as it helps to correctly interpret and extract information from text.

• Organizers: Mohammed Khalilia (Birzeit University), Imed Zitouni (Google), Mustafa Jarrar (Birzeit University), Tamer Elsayed (Qatar University), Sanad Malaysha (Birzeit University), Ala' Jabari (Birzeit University), Reem Suwaileh (Hamad Bin Khalifa University)

• For more information, please visit the shared task website: https://sina.birzeit.edu/nlu_sharedtask2024/ 

Shared Task 7: NADI 2024: Nuanced Arabic Dialect Identification

• • Description: Previous runs of NADI focused mainly on Dialect Identification and other NLP tasks targeting the advancement of Arabic NLP such as Machine Translation, which was introduced last year. The NADI 2024 shared task comprises three subtasks: multi-label country-level dialect identification (Subtask 1), level of dialectness estimation (Subtask 2), in addition to machine translation from four Arabic dialects to MSA (Subtask 3).

  • Organizers: Muhammad Abdul-Mageed (The University of British Columbia and MBZUAI), Chiyu Zhang (The University of British Columbia), Amr Keleg (The University of Edinburgh), AbdelRahim Elmadany (The University of British Columbia), Injy Hamed (New York University Abu Dhabi), Walid Magdy (The University of Edinburgh), Houda Bouamor (Carnegie Mellon University Qatar), Nizar Habash (New York University Abu Dhabi)

  • For more information, please visit the shared task website: https://nadi.dlnlp.ai/ 

Shared Task 8: KSAA-CAD Shared Task:  Contemporary Arabic Reverse Dictionary and Word Sense Disambiguation

• • Description: This shared task comprises two tasks: RD and WSD. The RD task focuses on identifying word embeddings that most accurately match a given definition, termed a "gloss," in Arabic. Conversely, the WSD task involves determining the specific meaning of a word in context, particularly when the word has multiple meanings. KSAA-CAD presents novel directions for researchers to investigate and offer significant contributions to the discipline.

  • Organizers: Waad Alshammari, Rawan Almatham, Amal Almazrua, Asma Al Wazrah, Muneera Alhoshan, Afrah Altamimi, Abdullah Alfaifi, Abdulrahman AlOsaimy

  • For more information, please visit the shared task website: https://arai.ksaa.gov.sa/sharedTask2024/ 

Important Dates

• May 10, 2024: Shared task papers due date

• June 17, 2024: Notification of acceptance

• July 1, 2024: Camera-ready papers due

• August 16, 2024: ArabicNLP conference
  All deadlines are 11:59 pm UTC -12h (“Anywhere on Earth”).

For any questions, please contact the Shared Task Chair: arabicnlp-shared-task-chair@sigarab.org

The Second Call for Shared Task Proposals

The Second Arabic Natural Language Processing Conference (ArabicNLP 2024) 

Co-located with ACL 2024 in Bangkok, Thailand, August 16, 2024. (Hybrid Mode).

We invite proposals for shared tasks related to Arabic NLP to be part of the ArabicNLP 2024 conference.

The proposals should provide an overview of the proposed task, motivation, data/resource collection and creation, task description, pilot run details (if available), a tentative timeline that matches the submission dates below, and task organizers (name, email, affiliation). Proposals in PDF format can be up to 4 pages.

Shared Task Proposal Submission URL: https://shorturl.at/eCJOS

Selection Process

The proposals will be reviewed by the organizing committee and selected based on multiple factors such as the novelty of the task, the expected interest from the community, how convincing the data collection plans are, the soundness of the evaluation method, and the expected impact of the task.

Task Organization

Upon acceptance, the task organizers are expected to verify that the task organization and data delivery to participants are happening in a timely manner, provide the participants with all needed resources related to the task, create a mailing list, and maintain communication and support to participants, create and manage CodaLab or similar competition website, manage submissions to CodaLab, write a task description paper, manage participants submissions of system description papers, and review and maintain the quality of submitted system description papers.

Important Dates for Shared Task Proposals

• January 23, 2024: Submission of shared tasks proposals due date

• February 6, 2024: Notification of acceptance of shared tasks