Experimental IR Meets Multilinguality

Multimodality and Interaction 9th

International Conference of the CLEF
Association CLEF 2018 Avignon France
September 10 14 2018 Proceedings
Patrice Bellot
Visit to download the full and correct content document:
https://textbookfull.com/product/experimental-ir-meets-multilinguality-multimodality-an
d-interaction-9th-international-conference-of-the-clef-association-clef-2018-avignon-fr
ance-september-10-14-2018-proceedings-patrice-bellot/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Reachability Problems 12th International Conference RP

2018 Marseille France September 24 26 2018 Proceedings
Igor Potapov

https://textbookfull.com/product/reachability-problems-12th-
international-conference-rp-2018-marseille-france-
september-24-26-2018-proceedings-igor-potapov/

Reversible Computation 10th International Conference RC

2018 Leicester UK September 12 14 2018 Proceedings
Jarkko Kari

https://textbookfull.com/product/reversible-computation-10th-
international-conference-rc-2018-leicester-uk-
september-12-14-2018-proceedings-jarkko-kari/

Exploring Service Science 9th International Conference

IESS 2018 Karlsruhe Germany September 19 21 2018
Proceedings Gerhard Satzger

https://textbookfull.com/product/exploring-service-science-9th-
international-conference-iess-2018-karlsruhe-germany-
september-19-21-2018-proceedings-gerhard-satzger/

Belief Functions Theory and Applications 5th

International Conference BELIEF 2018 Compiègne France
September 17 21 2018 Proceedings Sébastien Destercke

https://textbookfull.com/product/belief-functions-theory-and-
applications-5th-international-conference-belief-2018-compiegne-
france-september-17-21-2018-proceedings-sebastien-destercke/
Text Speech and Dialogue 21st International Conference
TSD 2018 Brno Czech Republic September 11 14 2018
Proceedings Petr Sojka

https://textbookfull.com/product/text-speech-and-dialogue-21st-
international-conference-tsd-2018-brno-czech-republic-
september-11-14-2018-proceedings-petr-sojka/

Web Information Systems and Applications: 15th

International Conference, WISA 2018, Taiyuan, China,
September 14–15, 2018, Proceedings Xiaofeng Meng

https://textbookfull.com/product/web-information-systems-and-
applications-15th-international-conference-wisa-2018-taiyuan-
china-september-14-15-2018-proceedings-xiaofeng-meng/

Business Process Management: 16th International

Conference, BPM 2018, Sydney, NSW, Australia, September
9–14, 2018, Proceedings Mathias Weske

https://textbookfull.com/product/business-process-
management-16th-international-conference-bpm-2018-sydney-nsw-
australia-september-9-14-2018-proceedings-mathias-weske/

Machine Learning and Knowledge Discovery in Databases

European Conference ECML PKDD 2018 Dublin Ireland
September 10 14 2018 Proceedings Part II Michele
Berlingerio
https://textbookfull.com/product/machine-learning-and-knowledge-
discovery-in-databases-european-conference-ecml-pkdd-2018-dublin-
ireland-september-10-14-2018-proceedings-part-ii-michele-
berlingerio/

Machine Learning and Knowledge Discovery in Databases

European Conference ECML PKDD 2018 Dublin Ireland
September 10 14 2018 Proceedings Part III Ulf Brefeld

https://textbookfull.com/product/machine-learning-and-knowledge-
discovery-in-databases-european-conference-ecml-pkdd-2018-dublin-
ireland-september-10-14-2018-proceedings-part-iii-ulf-brefeld/
 Patrice Bellot · Chiraz Trabelsi · Josiane Mothe
Fionn Murtagh · Jian Yun Nie · Laure Soulier
Eric SanJuan · Linda Cappellato · Nicola Ferro (Eds.)

Experimental IR Meets
Multilinguality,
LNCS 11018

Multimodality,
and Interaction
9th International Conference of the CLEF Association, CLEF 2018
Avignon, France, September 10–14, 2018
Proceedings

123
Lecture Notes in Computer Science 11018
Commenced Publication in 1973
Founding and Former Series Editors:
Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board
David Hutchison
Lancaster University, Lancaster, UK
Takeo Kanade
Carnegie Mellon University, Pittsburgh, PA, USA
Josef Kittler
University of Surrey, Guildford, UK
Jon M. Kleinberg
Cornell University, Ithaca, NY, USA
Friedemann Mattern
ETH Zurich, Zurich, Switzerland
John C. Mitchell
Stanford University, Stanford, CA, USA
Moni Naor
Weizmann Institute of Science, Rehovot, Israel
C. Pandu Rangan
Indian Institute of Technology Madras, Chennai, India
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Demetri Terzopoulos
University of California, Los Angeles, CA, USA
Doug Tygar
University of California, Berkeley, CA, USA
Gerhard Weikum
Max Planck Institute for Informatics, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/7409
Patrice Bellot Chiraz Trabelsi
•

Josiane Mothe Fionn Murtagh

•

Jian Yun Nie Laure Soulier

•

Eric SanJuan Linda Cappellato

•

Nicola Ferro (Eds.)

Experimental IR Meets
Multilinguality,
Multimodality,
and Interaction
9th International Conference
of the CLEF Association, CLEF 2018
Avignon, France, September 10–14, 2018
Proceedings

123
Editors
Patrice Bellot Laure Soulier
Aix-Marseille University Pierre and Marie Curie University
Marseille Cedex 20 Paris Cedex 05
France France
Chiraz Trabelsi Eric SanJuan
Virtual University of Tunis Université d’Avignon et des Pays de
Tunis Avignon
Tunisia France
Josiane Mothe Linda Cappellato
Systèmes d’informations, Big Data et Rec Department of Information Engineering
Institut de Recherche en Informatique de University of Padua
Toulouse Cedex 04 Padua, Padova
France Italy
Fionn Murtagh Nicola Ferro
Department of Computer Science University of Padua
University of Huddersfield Padua
Huddersfield Italy
UK
Jian Yun Nie
DIRO
Universite de Montreal
Montreal, QC
Canada

ISSN 0302-9743 ISSN 1611-3349 (electronic)

Lecture Notes in Computer Science
ISBN 978-3-319-98931-0 ISBN 978-3-319-98932-7 (eBook)
https://doi.org/10.1007/978-3-319-98932-7

Library of Congress Control Number: 2018950767

LNCS Sublibrary: SL3 – Information Systems and Applications, incl. Internet/Web, and HCI

© Springer Nature Switzerland AG 2018

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, express or implied, with respect to the material contained herein or for any errors or
omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
 Preface

Since 2000, the Conference and Labs of the Evaluation Forum (CLEF) has played a
leading role in stimulating research and innovation in the domain of multimodal and
multilingual information access. Initially founded as the Cross-Language Evaluation
Forum and running in conjunction with the European Conference on Digital Libraries
(ECDL/TPDL), CLEF became a standalone event in 2010 combining a peer-reviewed
conference with a multi-track evaluation forum. The combination of the scientific
program and the track-based evaluations at the CLEF conference creates a unique
platform to explore information access from different perspectives, in any modality and
language.
The CLEF conference has a clear focus on experimental information retrieval
(IR) as seen in evaluation forums (CLEF Labs, TREC, NTCIR, FIRE, MediaEval,
RomIP, TAC, etc.) with special attention to the challenges of multimodality, multi-
linguality, and interactive search ranging from unstructured, to semi-structured and
structured data. CLEF invites submissions on significant new insights demonstrated by
the use of innovative IR evaluation tasks or in the analysis of IR test collections and
evaluation measures, as well as on concrete proposals to push the boundaries of the
Cranfield/TREC/CLEF paradigm.
CLEF 20181 was jointly organized by Avignon, Marseille and Toulon Universities
and was hosted by the University of Avignon, France, during September 10–14, 2018.
The conference format consisted of keynotes, contributed papers, lab sessions, and
poster sessions, including reports from other benchmarking initiatives from around the
world.
The following scholars were invited to give a keynote talk at CLEF 2018: Gabriella
Pasi (University of Milano-Bicocca, Italia), Nicholas Belkin (Rutgers University, NJ,
USA), and Julio Gonzalo (UNED, Spain).
CLEF 2018 received a total of 39 submissions, of which a total of 13 papers (nine
long, four short) were accepted. Each submission was reviewed by three Program
Committee (PC) members, and the program chairs oversaw the reviewing and
follow-up discussions. In all, 13 different countries are represented in the accepted
papers. Many contributions this year tackle the medical e-Health and e-Health multi-
media retrieval challenges in different ways: from medical image analysis to query
suggestion. However, there are many other topics of research in the accepted papers
such as document clustering, social biases in IR, social book search, personality pro-
filing, to cite a few. As in previous editions since 2015, CLEF 2018 continued inviting
CLEF lab organizers to nominate a “best of the labs” paper that was reviewed as a full
paper submission to the CLEF 2018 conference according to the same review criteria
and PC. Among the nine invited papers, six were accepted as long and three as short.
Finally, eight posters were also accepted. Although they are not included in the LNCS

1
http://clef2018.clef-initiative.eu/.
VI Preface

volume, posters give the opportunity to their authors to discuss their research during the
conference and are accessible through the Web pages of the conference.
The conference integrated a series of workshops presenting the results of lab-based
comparative evaluations. CLEF 2018 was the ninth year of the CLEF Conference and
the 19th year of the CLEF initiative as a forum for IR Evaluation. The labs were
selected in peer review based on their innovation potential and the quality of the
resources created. The labs represented scientific challenges based on new data sets and
real-world problems in multimodal and multilingual information access. These data
sets provide unique opportunities for scientists to explore collections, to develop
solutions for these problems, to receive feedback on the performance of their solutions,
and to discuss the issues with peers at the workshops.
In addition to these workshops, the ten benchmarking labs reported results of their
year-long activities in overview talks and lab sessions. Overview papers describing
each of these labs are provided in this volume. The full details for each lab are
contained in a separate publication, the Working Notes, which are available online2.
The ten labs running as part of CLEF 2018 were as follows:
CENTRE@CLEF 2018 -CLEF/NTCIR/TREC Reproducibility3 aims to run a joint
CLEF/NTCIR/TREC task on challenging participants: (1) to reproduce the best results
of the best/most interesting systems in previous editions of CLEF/NTCIR/TREC by
using standard open source IR systems; (2) to contribute back to the community the
additional components and resources developed to reproduce the results in order to
improve existing open source systems.
CheckThat!4 aims to foster the development of technology capable of both spotting
and verifying check-worthy claims in political debates in English and Arabic.
Dynamic Search for Complex Tasks5: The lab strives to answer one key question:
How can we evaluate, and consequently build, dynamic search algorithms? The 2018
Lab focuses on the development of an evaluation framework, where participants submit
“querying agents” that generate queries to be submitted to a static retrieval system.
Effective “querying agents” can then simulate users toward developing dynamic search
systems.
CLEFeHealth6 provides scenarios that aim to ease patients, and nurses, under-
standing and accessing of e-Health information. The goals of the lab are to develop
processing methods and resources in a multilingual setting to enrich difficult-to-
understand e-Health texts, and provide valuable documentation. The tasks are: multi-
lingual information extraction; technologically assisted reviews in empirical medicine;
and patient-centered information retrieval.
ImageCLEF7 organizes three main tasks and a pilot task: (1) a caption prediction
task that aims at predicting the caption of a figure from the biomedical literature based

2
http://ceur-ws.org/Vol-2125/.
3
http://www.centre-eval.org/clef2018/.
4
http://alt.qcri.org/clef2018-factcheck/.
5
https://ekanou.github.io/dynamicsearch/.
6
https://sites.google.com/view/clef-ehealth-2018/.
7
http://www.imageclef.org/2018.
 Preface VII

only on the figure image; (2) a tuberculosis task that aims at detecting the tuberculosis
type, severity, and drug resistance from CT (computed tomography) volumes of the
lung; (3) a lifelog task (videos, images, and other sources) about daily activities
understanding and moment retrieval; and (4) a pilot task on visual question answering
where systems are tasked with answering medical questions.
LifeCLEF8 aims at boosting research on the identification of living organisms and
on the production of biodiversity data in general. Through its biodiversity
informatics-related challenges, LifeCLEF is intended to push the boundaries of the
state of the art in several research directions at the frontier of multimedia information
retrieval, machine learning, and knowledge engineering.
MC29 mainly focuses on developing processing methods and resources to mine the
social media (SM) sphere surrounding cultural events such as festivals, music, books,
movies, and museums. Following previous editions (CMC 2016 and MC2 2017), the
2018 edition focused on argumentative mining and multilingual cross SM search.
PAN10 is a networking initiative for digital text forensics, where researchers and
practitioners study technologies that analyze texts with regard to originality, authorship,
and trustworthiness. PAN offered three tasks at CLEF 2018 with new evaluation
resources consisting of large-scale corpora, performance measures, and Web services
that allow for meaningful evaluations. The main goal is to provide for sustainable and
reproducible evaluations, to get a clear view of the capabilities of state-of-the-art-
algorithms. The tasks are: author identification; author profiling; and, author obfuscation.
Early Risk Prediction on the Internet (eRisk)11 explores issues of evaluation
methodology, effectiveness metrics, and other processes related to early risk detection.
Early detection technologies can be employed in different areas, particularly those
related to health and safety. For instance, early alerts could be sent when a predator
starts interacting with a child for sexual purposes, or when a potential offender starts
publishing antisocial threats on a blog, forum, or social network. Our main goal is to
pioneer a new interdisciplinary research area that would be potentially applicable to a
wide variety of situations and to many different personal profiles. eRisk 2018 had two
campaign-style tasks: early detection of signs of depression and early detection of signs
of anorexia.
Personalized Information Retrieval at CLEF (PIR-CLEF)12 provides a framework
for the evaluation of personalized information retrieval (PIR). Current approaches to
the evaluation of PIR are user-centric, mostly based on user studies, i.e., they rely on
experiments that involve real users in a supervised environment. PIR-CLEF aims to
develop and demonstrate a methodology for the evaluation of personalized search that
enables repeatable experiments. The main aim is to enable research groups working on
PIR to both experiment with and provide feedback on the proposed PIR evaluation
methodology.

8
http://www.lifeclef.org/.
9
https://mc2.talne.eu/.
10
http://pan.webis.de/.
11
http://early.irlab.org/.
12
http://www.ir.disco.unimib.it/pir-clef2018/.
VIII Preface

Avignon is famous for its medieval architecture and its international theater festival.
The social program of CLEF 2018 set up a Science and Music Festival in medieval
downtown at Theâtre des Halles13 and surrounding gardens from Tuesday to Thursday.
Music is a very popular hobby among members of the scientific community. Evenings
were a mix of music and participatory science around PlantNet, OpenStreetMaps, and
Wikipedia. Tuesday was especially devoted to welcoming students at CLEF. On
Wednesday the focus was on IR scientific societies around the world mixing all CLEF
languages in one evening. Finally, science outreach activities were carried out on
Thursday; local musicians and students looking for a good time were invited to come
and meet the participants of the CLEF conference.
The success of CLEF 2018 would not have been possible without the huge effort of
several people and organizations, including the CLEF Association14, the PC, the Lab
Organizing Committee, the local organization committee in Avignon, the reviewers,
and the many students and volunteers who contributed.

July 2018 Patrice Bellot

Chiraz Trabelsi
Josiane Mothe
Fionn Murtagh
Jian Yun Nie
Laure Soulier
Eric Sanjuan
Linda Cappellato
Nicola Ferro

13
http://www.theatredeshalles.com/.
14
http://www.clef-initiative.eu/association.
 Organization

CLEF 2018, Conference and Labs of the Evaluation Forum – Experimental IR meets
Multilinguality, Multimodality, and Interaction, was hosted by the University of
Avignon and jointly co-organized by Avignon, Marseille and Toulon Universities,
France.

General Chairs
Patrice Bellot Aix-Marseille Université - CNRS LSIS, France
Chiraz Trabelsi University of Tunis El Manar, Tunisia

Program Chairs
Josiane Mothe SIG, IRIT, France
Fionn Murtagh University of Huddersfield, UK

Lab Chairs
Jian Yun Nie DIRO, Université de Montréal, Canada
Laure Soulier LIP6, UPMC, France

Proceedings Chairs
Linda Cappellato University of Padua, Italy
Nicola Ferro University of Padua, Italy

Publicity Chair
Adrian Chifu Aix-Marseille Université - CNRS LSIS, France

Science Outreach Program Chairs

Aurelia Barriere UAPV, France
Mathieu FERYN UAPV, France

Sponsoring Chair
Malek Hajjem UAPV, France
X Organization

Local Organization
Eric SanJuan (Chair) LIA, UAPV, France
Tania Jimenez (Co-chair) LIA, UAPV, France
Sebastien Fournier Aix-Marseille Université - CNRS LIS, France
Hervé Glotin Université de Toulon - CNRS LIS, France
Vincent Labatut LIA, UAPV, France
Elisabeth Murisasco Université de Toulon - CNRS LIS, France
Magalie Ochs Aix-Marseille Université - CNRS LIS, France
Juan-Manuel LIA, UAPV, France
Torres-Moreno
 Organization XI

Sponsors

Association francophone de Recherche d’Information et Applications

 CLEF Steering Committee

Steering Committee Chair

Nicola Ferro University of Padua, Italy

Deputy Steering Committee Chair for the Conference

Paolo Rosso Universitat Politècnica de València, Spain

Deputy Steering Committee Chair for the Evaluation Labs

Martin Braschler Zurich University of Applied Sciences, Switzerland

Members
Khalid Choukri Evaluations and Language resources Distribution
Agency (ELDA), France
Paul Clough University of Sheffield, UK
Norbert Fuhr University of Duisburg-Essen, Germany
Lorraine Goeuriot Université Grenoble Alpes, France
Julio Gonzalo National Distance Education University (UNED),
Spain
Donna Harman National Institute for Standards and Technology
(NIST), USA
Djoerd Hiemstra University of Twente, The Netherlands
Evangelos Kanoulas University of Amsterdam, The Netherlands
Birger Larsen University of Aalborg, Denmark
Séamus Lawless Trinity College Dublin, Ireland
Mihai Lupu Vienna University of Technology, Austria
Josiane Mothe IRIT, Université de Toulouse, France
Henning Müller University of Applied Sciences Western Switzerland
(HES-SO), Switzerland
Maarten de Rijke University of Amsterdam UvA, The Netherlands
Giuseppe Santucci Sapienza University of Rome, Italy
Jacques Savoy University of Neuchatel, Switzerland
Christa Womser-Hacker University of Hildesheim, Germany
 CLEF Steering Committee XIII

Past Members
Jaana Kekäläinen University of Tampere, Finland
Carol Peters ISTI, National Council of Research (CNR), Italy
(Steering Committee Chair 2000–2009)
Emanuele Pianta Centre for the Evaluation of Language
and Communication Technologies (CELCT), Italy
Alan Smeaton Dublin City University, Ireland
 Contents

Full Papers

Deep Multimodal Classification of Image Types in Biomedical

Journal Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Vincent Andrearczyk and Henning Müller

Multi-view Personality Profiling Based on Longitudinal Data . . . . . . . . . . . . 15

Kseniya Buraya, Aleksandr Farseev, and Andrey Filchenkov

Using R Markdown for Replicable Experiments in Evidence

Based Medicine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Giorgio Maria Di Nunzio and Federica Vezzani

Rethinking the Evaluation Methodology of Authorship

Verification Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Oren Halvani and Lukas Graner

Learning-to-Rank and Relevance Feedback for Literature Appraisal

in Empirical Medicine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Athanasios Lagopoulos, Antonios Anagnostou, Adamantios Minas,
and Grigorios Tsoumakas

Combining Tags and Reviews to Improve Social Book

Search Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Messaoud Chaa, Omar Nouali, and Patrice Bellot

Fast and Simple Deterministic Seeding of KMeans

for Text Document Clustering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Ehsan Sherkat, Julien Velcin, and Evangelos E. Milios

Medical Image Classification with Weighted Latent Semantic

Tensors and Deep Convolutional Neural Networks . . . . . . . . . . . . . . . . . . . 89
Spyridon Stathopoulos and Theodore Kalamboukis

Effects of Language and Terminology of Query Suggestions

on the Precision of Health Searches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
Carla Teixeira Lopes and Cristina Ribeiro
XVI Contents

Short Papers

Automatic Query Selection for Acquisition and Discovery

of Food-Drug Interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Georgeta Bordea, Frantz Thiessard, Thierry Hamon, and Fleur Mougin

Addressing Social Bias in Information Retrieval . . . . . . . . . . . . . . . . . . . . . 121

Jahna Otterbacher

Analyzing and Visualizing Translation Patterns of Wikidata Properties . . . . . 128

John Samuel

Character N-Grams for Detecting Deceptive Controversial Opinions . . . . . . . 135

Javier Sánchez-Junquera, Luis Villaseñor-Pineda,
Manuel Montes-y-Gómez, and Paolo Rosso

Best of CLEF 2017 Labs

Simply the Best: Minimalist System Trumps Complex Models

in Author Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Angelo Basile, Gareth Dwyer, Maria Medvedeva, Josine Rawee,
Hessel Haagsma, and Malvina Nissim

Textured Graph-Based Model of the Lungs: Application on Tuberculosis

Type Classification and Multi-drug Resistance Detection . . . . . . . . . . . . . . . 157
Yashin Dicente Cid, Kayhan Batmanghelich, and Henning Müller

Plant Classification Based on Gated Recurrent Unit. . . . . . . . . . . . . . . . . . . 169

Sue Han Lee, Yang Loong Chang, Chee Seng Chan, Joly Alexis,
Pierre Bonnet, and Herve Goeau

Microblog Contextualization: Advantages and Limitations

of a Multi-sentence Compression Approach . . . . . . . . . . . . . . . . . . . . . . . . 181
Elvys Linhares Pontes, Stéphane Huet,
and Juan-Manuel Torres-Moreno

Early Detection of Depression Based on Linguistic Metadata Augmented

Classifiers Revisited: Best of the eRisk Lab Submission . . . . . . . . . . . . . . . 191
Marcel Trotzek, Sven Koitka, and Christoph M. Friedrich

Deep Learning for ICD Coding: Looking for Medical Concepts

in Clinical Documents in English and in French . . . . . . . . . . . . . . . . . . . . . 203
Zulfat Miftahutdinov and Elena Tutubalina

Hierarchical Clustering Analysis: The Best-Performing Approach

at PAN 2017 Author Clustering Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Helena Gómez-Adorno, Carolina Martín-del-Campo-Rodríguez,
Grigori Sidorov, Yuridiana Alemán, Darnes Vilariño, and David Pinto
 Contents XVII

Attention-Based Medical Caption Generation with Image Modality

Classification and Clinical Concept Mapping . . . . . . . . . . . . . . . . . . . . . . . 224
Sadid A. Hasan, Yuan Ling, Joey Liu, Rithesh Sreenivasan,
Shreya Anand, Tilak Raj Arora, Vivek Datla, Kathy Lee,
Ashequl Qadir, Christine Swisher, and Oladimeji Farri

A Compound Model for Consumer Health Search . . . . . . . . . . . . . . . . . . . . 231

Hua Yang and Teresa Gonçalves

CLEF 2018 Lab Overviews

Overview of CENTRE@CLEF 2018: A First Tale in the Systematic

Reproducibility Realm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
Nicola Ferro, Maria Maistro, Tetsuya Sakai, and Ian Soboroff

Overview of LifeCLEF 2018: A Large-Scale Evaluation of Species

Identification and Recommendation Algorithms in the Era of AI. . . . . . . . . . 247
Alexis Joly, Hervé Goëau, Christophe Botella, Hervé Glotin,
Pierre Bonnet, Willem-Pier Vellinga, Robert Planqué,
and Henning Müller

Overview of PAN 2018: Author Identification, Author Profiling,

and Author Obfuscation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Efstathios Stamatatos, Francisco Rangel, Michael Tschuggnall,
Benno Stein, Mike Kestemont, Paolo Rosso, and Martin Potthast

Overview of the CLEF eHealth Evaluation Lab 2018 . . . . . . . . . . . . . . . . . 286

Hanna Suominen, Liadh Kelly, Lorraine Goeuriot, Aurélie Névéol,
Lionel Ramadier, Aude Robert, Evangelos Kanoulas, Rene Spijker,
Leif Azzopardi, Dan Li, Jimmy, João Palotti, and Guido Zuccon

CLEF MC2 2018 Lab Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

Malek Hajjem, Jean Valére Cossu, Chiraz Latiri, and Eric SanJuan

Overview of ImageCLEF 2018: Challenges, Datasets and Evaluation. . . . . . . 309

Bogdan Ionescu, Henning Müller, Mauricio Villegas,
Alba García Seco de Herrera, Carsten Eickhoff, Vincent Andrearczyk,
Yashin Dicente Cid, Vitali Liauchuk, Vassili Kovalev, Sadid A. Hasan,
Yuan Ling, Oladimeji Farri, Joey Liu, Matthew Lungren,
Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler,
Liting Zhou, Mathias Lux, and Cathal Gurrin

Evaluation of Personalised Information Retrieval

at CLEF 2018 (PIR-CLEF) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
Gabriella Pasi, Gareth J. F. Jones, Keith Curtis, Stefania Marrara,
Camilla Sanvitto, Debasis Ganguly, and Procheta Sen
XVIII Contents

Overview of eRisk: Early Risk Prediction on the Internet . . . . . . . . . . . . . . . 343

David E. Losada, Fabio Crestani, and Javier Parapar

Overview of the CLEF Dynamic Search Evaluation Lab 2018 . . . . . . . . . . . 362

Evangelos Kanoulas, Leif Azzopardi, and Grace Hui Yang

Overview of the CLEF-2018 CheckThat! Lab on Automatic

Identification and Verification of Political Claims . . . . . . . . . . . . . . . . . . . . 372
Preslav Nakov, Alberto Barrón-Cedeño, Tamer Elsayed,
Reem Suwaileh, Lluís Màrquez, Wajdi Zaghouani,
Pepa Atanasova, Spas Kyuchukov, and Giovanni Da San Martino

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

Full Papers
 Deep Multimodal Classification of Image
Types in Biomedical Journal Figures

Vincent Andrearczyk1(B) and Henning Müller1,2

1
University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
vincent.andrearczyk@hevs.ch
2
University of Geneva (UNIGE), Geneva, Switzerland

Abstract. This paper presents a robust method for the classification

of medical image types in figures of the biomedical literature using the
fusion of visual and textual information. A deep convolutional network
is trained to discriminate among 31 image classes including compound
figures, diagnostic image types and generic illustrations, while another
shallow convolutional network is used for the analysis of the captions
paired with the images. Various fusion methods are analyzed as well
as data augmentation approaches. The proposed system is validated on
the ImageCLEF 2013 and 2016 figure and subfigure classification tasks,
largely improving the currently best performance from 83.5% to 93.7%
accuracy and 88.4% to 89.0% respectively.

1 Introduction
The information contained in an image and the methods employed to extract
it largely differ depending on its modality, making the latter a crucial aspect
of medical image analysis and retrieval. An image type classification is, there-
fore, a useful preliminary filtering step prior to further analysis [2,16]. Besides
this, the modality is a relevant information to be determined for medical image
or document retrieval, allowing clinicians to filter their search by a particular
modality, often specific to a pathology or organ of interest. Various modality
classification tasks, among others, have been released through the ImageCLEF
challenges [7,10]. We focus this work on the 2013 and 2016 ImageCLEF modality
classification tasks, as they offer multimodal text and image data. The database
is publicly available and the results are fully reproducible as a consequence. The
database also originates from the PubMed Central database (it is a small sub-
set of PubMed Central), allowing us to classify this large database for further
processing and analysis. Much of the medical knowledge is stored in the medical
literature, for example in the form of images and text, although the image type
information is not available. Making this content accessible for research can help
in many other tasks, such as retrieval or classification.
Multimodal analysis is commonly used to extract and fuse information from
multiple modalities [11,24]. In this work, images and captions contain comple-
mentary information fused to boost the classification accuracy. Many methods
c Springer Nature Switzerland AG 2018
P. Bellot et al. (Eds.): CLEF 2018, LNCS 11018, pp. 3–14, 2018.
https://doi.org/10.1007/978-3-319-98932-7_1
4 V. Andrearczyk and H. Müller

have been used to extract high-level features from text and images independently
and to fuse them. Convolutional Neural Networks (CNNs) have obtained state
of the art performance in most computer vision [5,9] and biomedical image anal-
ysis [17] tasks. It is also well suited for text analysis [12]. This paper introduces
several late fusion methods to combine powerful visual and textual CNNs.

2 Related Work
Multimodal textual and visual analysis has been widely studied for applications
including annotation and captioning [11], image generation from text [24], text
and image feature fusion for retrieval and classification [7]. A total of 51 runs
from eight groups were presented in [7] for the ImageCLEF 2013 modality classi-
fication challenge. The best results (81.7% classification accuracy) were obtained
by visual and textual fusion from the IBM Multimedia Analytics group [1] (see
Table 1). A set of color and texture, local and global descriptors (including a color
histogram, moments, wavelets, Local Binary Patterns (LPB) and Scale-Invariant
Feature Transform (SIFT)) was extracted as visual descriptors and fused with
multiple textual descriptors. The best results were obtained using a maximum
late fusion with a classifier built on top of modality tailored keywords (with a
hand-selected vocabulary that likely improved the performance) and a two-level
Support Vector Machine (SVM) classification. The methods developed by other
teams reported in [7] include various types of similar hand-crafted visual and
textual descriptors combined by multiple fusion methods.
In [4], the authors build upon [1] to develop a more complex system. An
ensemble of SVM models is trained on top of similar visual features, while the
text is analyzed by scoring based on the detection of manually-selected patterns
from the captions and from sentences in the body of the article. A weighted score
average trained on a subset of the training data was used for fusing the visual
and textual information. The best current system reached an accuracy of 83.5%
on ImageCLEF 2013 modality classification.
Another set of hand-crafted visual and textual features are combined in [19].
The visual features include local and global texture and color features, while
Bag-of-Words (BoW) features are used to analyze the captions. Multiple CNNs
are combined in [23] by late fusion (average, maximum, majority and median),
yet the resulting accuracy is lower than the shallow hand-crafted methods in [7].
More recently in [16,22], pre-trained deep visual CNNs are finetuned and their
outputs are combined in an ensemble classifier with basic voting fusion meth-
ods. Besides this, deep pre-trained CNNs obtained better performance than shal-
lower ones, motivating the use of very deep pre-trained networks even on these
small datasets. A major drawback of combining multiple CNNs is the increase
of computational complexity and redundancy of features to obtain only a lim-
ited accuracy improvement (less than 1% in [22]). A multimodal approach based
on ensemble learning of various textual and visual features was proposed in
[15], obtaining the state of the art results on the ImageCLEF 2016 subfigure
modality classification task [8]. BoW textual features extracted from the arti-
cle’s text and captions are combined with visual features including hand-crafted
 Deep Multimodal Classification of Image Types 5

shallow texture and color features, Bag-of-Visual-Words (BoVW) and deep fea-
tures (ResNet152 [5]). The best results were obtained with an ensemble of SVMs
trained on all the features (see Table 3).

3 Methods
3.1 ImageCLEF 2013 Modality Dataset
The goal of this task is to classify the images into medical modalities and
other images types. Three main categories, namely compound figures, diagnostic
images and generic illustrations are divided into 31 sub-categories [7,18]. The
modality hierarchy and more details on the dataset can be found in [7]. A total
of 2879 training and 2570 test images are provided. The classes are highly imbal-
anced, reflecting the distribution of the images in the data (PubMed Central1 )
containing a large proportion of compound figures.

3.2 ImageCLEF 2016 Modality Subfigures Dataset

The goal of this task is similar to the task in ImageCLEF 2013, although the
images (subfigures) originate from a segmentation of larger compound figures.
The modality hierarchy is the same as in 2013 without the compound figure cat-
egory. The two main categories are diagnostic images and generic illustrations,
further divided into 30 sub-categories [8]. A total of 6776 training and 4166 test
images are provided. The captions are provided for the entire compound fig-
ures before segmentation and are therefore less specific than in the 2013 task.
Although the classes are less imbalanced than ImageCLEF 2013 since the “com-
pound figure” category is not present, the“GFIG” category (including statistical
figures, graphs and charts) is strongly overrepresented with approximately 44%
of the training samples while some classes contain less than 10 samples.

3.3 Overview of the Approach

An overview of the developed networks and fusion approaches is illustrated in
Fig. 1. The components are described in more details in the following sections.

3.4 Visual Analysis

DenseNet [9] is a CNN having each layer connected to every other layer (within
a dense block, see Fig. 1, right). The dense blocks are preceded by a 7 × 7 con-
volution layer and 3 × 3 max pooling. Each dense block is composed of densely
connected 1 × 1 and 3 × 3 convolutions. The transitions between blocks con-
tain a 1 × 1 convolution and 2 × 2 average pooling. All the convolutions are
regularized with batch normalization and activated by a ReLU. The last dense
block is globally down-sampled by a 7 × 7 average pooling and connected to a
1
https://www.ncbi.nlm.nih.gov/pmc/.
6 V. Andrearczyk and H. Müller

Fig. 1. Overview of the proposed deep learning visual and textual fusion method.

softmax activated dense layer with 30 or 31 neurons (number of classes). More

details can be found in [9]. This architecture obtained excellent results on vari-
ous image classification datasets while reducing the number of parameters and
computation (floating point operations) as compared to other commonly used
networks (e.g. AlexNet, VGG, GoogleNet and ResNet). On the proposed exper-
iments, DenseNet169 obtained the best results as compared to DenseNet121 [9],
ResNet50 and 152 [5], VGG19 [20], in line with recent computer vision and
biomedical image analysis results. The training data are limited (2789 or 6776
images without data augmentation) and transfer learning is required to obtain
a robust image classification. We use networks pre-trained on ImageNet and
replace the last fully connected softmax activated dense layer by a layer of 31
or 30 neurons, equivalent to the number of classes in the ImageCLEF 2013 and
2016 datasets respectively.
To increase the visual training data, we explore two complementary data aug-
mentation strategies. The first strategy uses extra training data from the other
task, i.e. using ImageCLEF 2016 data for training ImageCLEF 2013 and vice
versa. We also use images from the ImageCLEF 2016 compound figure detection
task for training ImageCLEF 2013 as the compound class is not represented in
the subfigure set. We ensure that no image is present twice in the training set or
in both the training and test sets. The second data augmentation strategy con-
sists of a set of random transformations applied to the training images including
horizontal and vertical flips, width and height shift in the range of [0, 0.1] of the
total width and height respectively and a rotation in the range [0◦ , 5◦ ].
 Deep Multimodal Classification of Image Types 7

3.5 Text Analysis

We develop a CNN on top of word embeddings as inspired by [12], in which a

CNN for sentence classification was developed. Our architecture is illustrated in
Fig. 1 (left). The words are embedded into a low (300) dimensional space using
fastText word embedding [3] pre-trained on Wikipedia data2 . Similar results were
obtained with a Global Vectors for word representation (GloVE) pre-trained on
Wikipedia 2014 and Gigaword 5, while pre-training on biomedical text performed
worse.
The maximum number of tokens was set to 20,000 and the maximum length
of a caption was set to 200. The embedding layer is finetuned together with
the network in order to adapt the embedding to this particular caption clas-
sification task and domain. We use convolution kernels ranging from 1 × 1 to
5 × 5 to perform a multi-scale analysis with scales that we deem relevant for the
caption task. We confirmed experimentally that adding larger kernels does not
improve the performance while significantly increasing the number of parame-
ters. Recurrent Neural Networks (RNNs) may seem more intuitive than CNNs
and better suited for natural language processing since local features captured
by convolution filters are not as evident in text as they are in images. Captions,
however, offer a relatively structured and controlled domain in which words are
often organized in meaningful local features. CNNs are also better at detecting
key phrases or combinations of words than RNNs, which is a useful asset for the
evaluated task since the modality is often described by a single sentence or a
group of words. The speed of convolution computations is an important aspect
in the choice of the architecture. We also experimented with a 1D convolutional
network, a Long Short-Term Memory (LSTM) network and a stacked LSTM
network, resulting in a lower accuracy.

3.6 Decision-Level Fusion

The decision-level fusion combines the visual and textual predictions. We first
train the visual and textual networks independently, then combine the class prob-
abilities, i.e. outputs of the softmax layers. Simple fusions are used including (a)
a weighted sum, (b) a maximum probability decision and (c) a product of proba-
bilities (elementwise product of probability vectors). Equation 1 summarizes the
class prediction of these three fusion methods.

csum = amax(αyv + (1 − α)yt ),

cmax = amax(max(yv , yt )), (1)
cprod = amax(yv ◦ yt ),

where csum , cmax , cprod are the class predictions from (a), (b) and (c), yv and yt
are the probability vectors of the visual and textual networks respectively. The
weight α ∈ [0, 1] is used to balance the importance of the visual and textual parts
2
https://dumps.wikimedia.org.
8 V. Andrearczyk and H. Müller

in (a). Another fusion method is to train a single layer Multi-Layer Perceptron

(MLP) on top of the prediction layer. We freeze all previous layers to train
only the last added layer. Using a two-layer MLP results in similar performance,
yet increases the complexity. We do not use artificial data augmentation when
training the MLP as the visual augmentation is learned when training DenseNet
individually and then the network requires visual and textual pairs.

3.7 Feature-Level Fusion

The feature-level method fuses the outputs of intermediate layers from the visual
and textual networks. We first train the two networks independently, then add
one layer on top of the last layer before the softmax activated one and train
similarly to the decision-level MLP. The networks’ deep representations, inputs
to the MLP, are richer and more complete than the decision-level ones (IR2560
and IR1664 for the textual and visual representations respectively, vs. IRN , with
N = 31, or N = 30 for the decision level).

4 Experimental Results
4.1 Network Setups
The networks are trained with an Adam [13] optimizer. The textual, visual and
fusion MLP networks are trained for N = 100, N = 25 and N = 50 epochs
respectively. The initial learning rate is set to 10−4 for finetuning the visual
network and 10−3 for the textual network and MLP from scratch, average decays
β1 and β2 are 0.9 and 0.999 respectively, the learning rate decay is 0.1
N and the
batch size 32. Due to the high class imbalance in the training set, class weights
are used during training for weighting the loss function as: wi = nmax /ni , where
nmax and ni are the number of training samples of the most represented class
and of class i respectively. The most represented class is the one with most
training samples, i.e. “compound figures” in ImageCLEF 2013 and “GFIG” in
ImageCLEF 2016. For the visual network, class weights are not needed when
artificial data augmentation is used.

4.2 Classification Results ImageCLEF 2013

The results are reported and compared with the best current systems in Table 13 .
Best results of the 51 runs submitted by eight groups [7] are reported as well
as the best results in the literature obtained after the challenge [4]. In [1,4],
vocabularies and text patterns were manually selected, and in [4], the text in
the body of the article was also used.
We do not use extra training data for the captions because in ImageCLEF
2016 the captions relate to the original compound figures from which the sub-
figures are segmented. Consequently, the textual network is trained with only
3
34 images were removed from the 2013 dataset since the original challenge due to
their presence in both training and test sets.
 Deep Multimodal Classification of Image Types 9

Table 1. Comparison of our methods with the best runs in ImageCLEF 2013.

Modality Method Accuracy

Textual IBM modality run1 [1] 64.2%
IBM textual [4] 69.6%
textual CNN 71.9%
Visual IBM modality run4 [1] 80.8%
IBM visual [4] 82.2%
DenseNet169 w/o data augm. w/o extra training 83.8%
DenseNet169 w/ data augm. w/o extra training 84.5%
DenseNet169 w/ data augm. w/ extra training 86.8%
Fusion IBM modality run8 [1] 81.7%
IBM fusion [4] 83.5%
Weighted sum fusion w/ extra training 89.2%
Maximum fusion w/ extra training 89.4%
Product fusion w/ extra training 91.8%
Decision-level MLP w/ extra training 86.0%
Feature-level MLP w/ extra training 93.7%

ImageCLEF 2013 training data. For the same reason, the MLPs are also trained
without extra training data in order to maintain pairs of visual and textual
inputs. However, the visual network is first trained with artificial data augmen-
tation and extra training data before being fused with the textual network.
The best fusion results are obtained with the feature-level MLP (93.7%).
The previously best results (IBM [4]) on ImageCLEF 2013 were obtained with-
out using extra data, yet our approach without extra data also outperforms
them (91.9% vs. 83.5%, not reported in Table 1). The weighted loss described in
Sect. 4.1 considerably improves the performance of our approach since the best
performance obtained without weighted loss is 92.7% (not reported in the table)
vs. 93.7% with. The confusion matrix of the best results (Feature-level MLP w/
extra training) is shown in Fig. 2.
The most relevant classes are the diagnostic images as they offer more poten-
tial in clinical applications such as retrieval. The confusion matrices for the
three main categories (compound, diagnostic and generic illustrations) are illus-
trated in Fig. 3. It shows that our approach performs an excellent discrimination
between diagnostic (e.g. MRI, CT, histopathology) and other images with lower
relevance (e.g. compound figures, diagrams and maps), which was of critical
importance for the development of the datasets in [6].
In order to evaluate the complementarity of the visual and textual infor-
mation, we measured the overlap of correct classification. With the best MLP
method previously described, the percentage of images correctly classified by
both the visual and textual networks is 64.3%. 22.5% of the test set is cor-
rectly classified by the visual network but incorrectly classified by the textual
10 V. Andrearczyk and H. Müller

Fig. 2. Normalized confusion matrix (%) of the feature-level fusion method on Image-
CLEF 2013. COMP stands for compound figures. Labels starting with D are diagnostic
modalities, those with G are generic illustrations. The complete list of labels can be
found in [7].

Fig. 3. Normalized confusion matrices for the three main categories in ImageCLEF
2013: compound, diagnostic and generic illustrations.
 Deep Multimodal Classification of Image Types 11

one and, vice-versa, 7.6% is correctly classified using the caption but incorrectly
classified using visual information. These results suggest, as confirmed by the
fusion results in Table 1, that the visual and textual analyses offer some degree
of complementarity to boost the final classification accuracy.
The accuracy obtained with multiple values of α in Eq. 1 is illustrated in
Fig. 4. The best results with this weighted sum fusion are obtained with a con-
tribution of the visual analysis slightly larger than the textual one (α = 0.51),
although a gradually reducing, yet neat, improvement from the single modality
results is obtained with α values in the range [0.51, 0.99].

Fig. 4. Accuracy of the sum fusion method for various weights α on ImageCLEF 2013.

The networks are implemented in Keras with TensorFlow backend and writ-
ten in Python. The computational training and test times are reported in Table 2
using a Titan Xp GPU.

Table 2. Computation time of the various networks on ImageCLEF 2013.

Method Train time (nb. images) Test time (nb. images)

Textual CNN 1,079 s (2789) 1.1 s (2570)
DenseNet169 w/o extra training 3,233 s (2789) 14.6 s (2570)
DenseNet169 w/ extra training 22,510 s (25880) 14.6 s (2570)
Feat.-level MLP w/ extra training 2,626 s (25880) 15.2 s (2570)

4.3 Classification Results ImageCLEF 2016

The results obtained on the ImageCLEF 2016 subfigure classification task are
reported and compared with the best current systems in Table 3.
Our visual analysis based on DenseNet trained with class weights outperforms
the complex ensemble classification of features in [15].
The textual analysis has less relevance in this challenge as the captions are
only available for the compound figures from which multiple subfigures origi-
nate and not precisely for each subfigure, as such a separation is not done sys-
tematically. The textual analysis still brings complementary information to the
12 V. Andrearczyk and H. Müller

Table 3. Comparison of our methods with the best runs in ImageCLEF 2016.

Modality Method Accuracy

Textual MLKD [21] 58.4%
BCSG [15] w/ extra training 72.2%
textual CNN 69.1%
textual CNN w/ extra training 72.2%
Visual IPL [21] 84.0%
BCSG [15] w/ extra training 85.4%
DenseNet169 w/o data augm. w/o extra training 86.2%
DenseNet169 w/ data augm. w/o extra training 86.8%
DenseNet169 w/ data augm. w/ extra training 87.9%
Fusion BCSG [15] w/ extra training 88.4%
weighted sum fusion w/ extra training 88.0%
maximum fusion w/ extra training 87.8%
product fusion w/ extra training 88.7%
Decision-level MLP w/ extra training 88.5%
Feature-level MLP w/ extra training 89.0%

visual analysis as shown by the fusion results. BCSG [15] makes use of the text
and caption for the textual analysis. A class distribution of the test set is also
learned based on the ImageCLEF 2015 dataset. We decided not to implement
such approaches for a better generalization to unknown data without prior on
the class distribution. Despite this simplified setup, our textual classifier is on a
par with [15] (72.2%). Extra training data are also used in [15].
The best fusion method is again obtained with a feature-level MLP (89.0%).
Slightly lower accuracy is obtained with the basic product fusion and decision-
level MLP (88.7% and 88.5%).

5 Discussions and Future Work

As illustrated in the experiments, the proposed approach largely outperforms

the state of the art [4] on the ImageCLEF 2013 dataset (93.7% vs. 83.5%).
The results on ImageCLEF 2016 also outperform the best results [15] (89.0%
vs. 88.4%). For this second experiment, our approach uses fewer data (only
captions for the textual part) and no prior on the test data classes distribution.
Besides this, our method jointly trains the visual and textural networks, whereas
[15] requires learning an ensemble of hand-crafted and trainable features with
multiple classifiers, which may limit its generalization and interpretability. The
reported results demonstrated the major importance of the visual analysis in
the developed method (86.8% and 87.5% accuracy), in line with the results and
conclusions from the literature [1,4,7,14]. The visual data augmentation had an
Another random document with
no related content on Scribd:
 5. Stamina a Pistillo diducta; antherâ unâ lente auctâ.
6. Stylus et Stigma lente aucta.

SPECIFIC CHARACTER.

Heath, with tips two-horned at their base, within the blossom; the leaves
grow by fours, and are rolled back; the flowers are clammy, grow in
bunches, are flask-shaped, striped, and nearly an inch and a half long.

DESCRIPTION.

Stem shrubby, grows about a foot high, and branching; the branches are
crooked, and thread-shaped.
Leaves grow by fours, are three-sided, fringed, rolled back, sharp-
pointed, smooth on the upper, and furrowed on the under surface; with short
footstalks, pressed to the branches.
Flowers grow in bunches, mostly of four, are clammy, and terminate the
branches; the fruit-stalks are long, having three spathula-shaped floral
leaves.
Empalement. Cup four-leaved, which are spathula-shaped, fringed at the
edges, clammy, and pressed to the blossom.
Blossom swelled at the base, and tapering to the top, striped
longitudinally, nearly an inch and a half long, straightened at the mouth,
which is of a deep purple; the segments egg-shaped, and spreading.
Chives. Eight hair-like threads; tips within the blossom, are cleft in two,
and two-horned at their base.
Pointal. Seed-vessel club-shaped, and furrowed. Shaft without the
blossom, and thread-shaped. Summit four-cornered, and green.
Native of the Cape of Good Hope.
Flowers from the month of July till October.

REFERENCE.
1. A Leaf magnified.
2. The Empalement and Blossom.
3. The Empalement magnified.
4. The Chives and Pointal.
5. The Chives detached from the Pointal; one tip magnified.
6. The Shaft and its Summit magnified.
 ERICA baccans.

CHARACTER SPECIFICUS.

Erica, antheris cristatis, inclusis; folia quaterna, rigida, glauca; corollis

globosis, umbellatis calycibusque purpureis,

DESCRIPTIO.

Caulis fruticosus, erectus, sesquipedalis, ramosissimus; rami rigidi,

erecti, torti.
Folia quaterna, glauca, incurvata, subtus sulcata, rigida, serrulata.
Flores terminales, plerumque quatuor, umbellati, nutantes; pedunculi
longi, purpurei, bracteis tribus instructi.
Calyx. Perianthium tetraphyllum, foliolis ovatis, carinatis, adpressis.
Corolla globosa purpurea, ima quadrata; limbo quadrilobo, laciniis
æqualibus, erectis.
Stamina. Filamenta octo capillaria, receptaculo inserta. Antheraæ
cristatæ, inclusæ.
Pistillum. Germen globosum, magnum, sulcatum; stylus subexsertus,
pyramidatus; stigma tetragonum.
Habitat ad Caput Bonæ Spei.
Floret a mense Aprilis, in Julium.

REFERENTIA.

1. Calyx, et Corolla,
2. Calyx lente auctus.
3. Stamina, et Pistillum.
4. Stamina a Pistillo diducta, anthera una lente aucta.
5. Pistillum et Germen lente auctum.
 SPECIFIC CHARACTER.

Heath, with crested tips, within the blossom; the leaves grow by fours, are
harsh, and of a whitish green; the flowers are globular, in bunches, and are,
as well as their cups, purple.

DESCRIPTION.

Stem shrubby, upright, a foot and a half high, and very branching; the
branches are harsh, upright, and twisted.
Leaves grow by fours, are of a whitish green, and bent inwards, furrowed
beneath, harsh, and slightly sawed at the edge.
Flowers grow at the ends of the branches, mostly by fours, in bunches,
and nodding; the foot-stalks are long and purple, having three floral leaves.
Empalement. Cup four-leaved, leaflets egg-shaped, keeled, and pressed
to the blossom.
Blossom globular, purple, and squared at the base; the border four-lobed,
the segments of which are upright, and equal.
Chives. Eight hair-like threads fixed into the receptacle. The tips crested,
and within the blossom.
Pointal. Seed-vessel globular, large, and furrowed; the shaft nearly
without the blossom, tapering from the base; the summit four-cornered.
Native of the Cape of Good Hope.
Flowers from April, till July.

REFERENCE.

1. The Empalement, and Blossom.

2. The Empalement magnified.
3. The Chives and Pointal.
4. The Chives detached from the Pointal; one tip magnified.
5. The Pointal and Seed-vessel magnified.
 ERICA Banksia.

CHARACTER SPECIFICUS.

Erica, antheræ muticæ, exsertæ, attenuata in filamenta plana; corolla

cylindrica, viridi-lutea; caulis decumbens, sub-spithamæus; folia terna.

DESCRIPTIO.

Caulis decumbens, rigidus, scaber, sub-spithamæus, ramosissimus;

ramulis brevibus.
Folia terna, subulata, trigona, acuminata, subtus tenuissime sulcata,
supra plana, glabra, profundé viridia; petiolis brevissimis, adpressis.
Flores in ultimis ramulis, bini vel terni, cernui; pedunculi brevissimi,
bracteis nullis.
Calyx. Perianthium duplex, exterius triphyllum, foliolis lato-ovatis,
rigidis, carinatis, acuminatis; interius tetraphyllum, foliolis flaccidis,
linearibus, longioribus.
Corolla cylindrica, ad basin parum inflata, viridi-lutea, oris laciniis
revolutis.
Stamina. Filamenta octo linearia, plana, corolla multoties longiora,
receptaculo inserta. Antheræ muticæ, exsertæ, nec filamentis distinguendis.
Pistillum. Germen sub-ovatum. Stylus filiformis, exsertus. Stigma
tetragonum.
Habitat ad Caput Bonæ Spei.
Floret a mensi Februarii, in Julium.

REFERENTIA.

1. Calyx, et Corolla,
2. Calyx lente auctus.
 3. Stamina, et Pistillum.
4. Stamina a Pistillo diducta; anthera una lente aucta.
5. Stylus, et Stigma, lente aucta.

SPECIFIC CHARACTER.

Heath, whose tips are beardless, without the blossom, and tapering into
threads, which are flat; the blossom is cylindrical, of a greenish yellow; the
stem droops, is scarce a span high; the leaves grow by threes.

DESCRIPTION.

Stem drooping, harsh, rough, less than a span in height, and very much
branched; the smaller branches are short.
The Leaves grow by threes, are awl-shaped, three-sided, and sharp-
pointed, slightly channelled on the under part, smooth on the upper, and flat,
of a deep green colour, having short foot-stalks pressed to the stem.
Flowers grow by twos or threes at the end of the smaller branches,
hanging down; the foot-stalks are very short, without any floral leaves.
Empalement. Cup double, the outer three-leaved, which are of a broad
egg-shape, harsh, keeled, and sharp pointed; the inner is four-leaved, longer
than the former, linear, and limber.
The Blossom is cylindrical, slightly swelled at the base, of a greenish-
yellow; the segments of the mouth are rolled back.
Chives. Eight linear, flat threads, much longer than the blossom, fixed
into the receptacle. Tips beardless, without the blossom, and not to be
distinguished from the threads.
Pointal. Seed-vessel nearly oval. Shaft thread-shaped, and without the
blossom. Summit four-cornered.
Native of the Cape of Good Hope.
Flowers from February, till July.

REFERENCE.
1. The Empalement, and Blossom.
2. The Empalement magnified.
3. The Chives, and Pointal.
4. The Chives detached from the Pointal; one tip magnified.
5. The Shaft, and its Summit, magnified.
 ERICA Bruniades.

CHARACTER SPECIFICUS.

Erica, antheris muticis, exsertis, floribus terminalibus, sessilibus, sub-

ternis; corollis albidis, urceolatis, tectis calyce lanato; calycis foliolis ovatis,
carneis; foliis ternis, linearibus, obtusis, margine revolutis, pilosis.

DESCRIPTIO.

Caulis filiformis, sub-erectus, glaber, flexuosus; rami filiformes, villosi,

ramuli capillares, frequentes, tomentosi.
Folia terna, linearia, obtusa; seniora, reflexa, juniora patentia, margine
pilis longis hirta.
Flores in extremis ramulis terminales, sessiles, sub-terni; bracteis tribus,
minutis, adpressis.
Calyx. Perianthium tetraphyllum, foliolis ovatis, concavis, exterioribus
totis densissime albo-lanatis, interioribus carneis.
Corolla urceolata, pubescens, alba, tota tecta calyce lanato; oris laciniis
patentibus, obtusis.
Stamina. Filamenta octo capillaria. Antheris muticis, exsertis, atris.
Pistillum. Germen turbinatum, pubescentem, ad basin glandulosum.
Stylus filiformis, staminibus paulo longior. Stigma tetragonum.
Habitat ad Caput Bonæ Spei.
Floret a mensi Februarii in Junium.

REFERENTIA.

1. Folium unum, lente auctum.

2. Calyx.
3. Calyx, auctus.
 4. Corolla.
5. Corolla, aucta.
6. Stamina, et Pistillum.
7. Stamina a Pistillo diducta.
8. Stamen unum, auctum.
9. Pistillum, auctum.

SPECIFIC CHARACTER.

Heath, with beardless tips, without the blossom, flowers terminating the
smaller branches, nearly without foot-stalks, mostly by threes; blossoms
white, pitcher-shaped, and covered by a woolly cup; the leaflets of the cup
are egg-shaped, and flesh-coloured; the leaves grow by threes, are linear,
blunt, rolled back at the edge, and hairy.

DESCRIPTION.

Stem thread-shaped, nearly upright, smooth, and growing zig-zag; the

branches are thread-shaped, and hairy; the smaller branches are hair-like,
numerous, and downy.
Leaves grow by threes, are linear, and blunt, the older ones are bent back,
the younger ones spreading out, the edge covered with long hairs.
Flowers grow at the extremity of the younger branches, sitting close,
and mostly by threes; three small floral leaves, pressed to the cup.
Empalement. Cup of four leaves, which are egg-shaped, concave, on
their outsides covered with a thick white wool, and on their insides flesh
coloured.
Blossom pitcher-shaped, downy, white, and covered totally by a woolly
cup; the segments of the mouth are spread out, and blunt.
Chives. Eight hair-like threads. Tips beardless, without the blossom, and
black.
Pointal. Seed-bud turban-shape, downy, and glandular at the base. Shaft
thread-shaped, a little longer than the chives. Summit four-cornered.
Native of the Cape of Good Hope.
Flowers from February, till June.

REFERENCE.

1. A Leaf, magnified.
2. The Empalement.
3. The Empalement, magnified.
4. The Blossom.
5. The Blossom, magnified.
6. The Chives, and Pointal.
7. The Chives, detached from the Pointal.
8. A Chive, magnified.
9. The Pointal, magnified.
 ERICA caffra.

CHARACTER SPECIFICUS.

Erica, antheris aristatis, inclusis, stylo exserto; corollis campanulatis,

albidis; floribus umbellatis, secundis, cernuis, odoratissimis; foliis quaternis,
linearibus, scabridis.

DESCRIPTIO.

Caulis fructicosus, pedalis, rami et ramuli erecti, cinereo-tomentosi,

rigidi.
Folia quaterna, linearia, obtusa, subciliata, scabrida, patentia; petiolis
brevissimis, adpressis.
Flores in ramis supremis racemosi, secundi, cernui, umbellati; pedunculi
brevissimi, bracteis tribus minutis instructi.
Calyx. Perianthium tetraphyllum, foliolis acuminatis, brevissimis,
glabris, adpressis,
Corolla campanulata, minuta, odoratissima, alba; laciniis erectis,
apicibus parum reflexis.
Stamina. Filamenta octo capillaria, apice curvata. Antheræ aristatæ,
inclusæ.
Pistillum. Germen sub-globosum, sulcatum. Stylus exsertus. Stigma
tetragonum.
Habitat ad Caput Bonæ Spei.
Floret a mensi Octobris, in Februarium.

REFERENTIA.

1. Folium unum, lente auctum.

2. Calyx, et Corolla.
 3. Calyx, lente auctus.
4. Stamina, lente aucta.
5. Pistillum, lente auctum.

SPECIFIC CHARACTER.

Heath, with bearded tips, within the blossom; the shaft without; the
blossoms bell-shaped and white; flowers grow in bunches, pointing all one
way, nodding and very sweet; leaves grow by fours, linear and rough.

DESCRIPTION.

Stem shrubby, growing a foot high, the larger and smaller branches grow
upright, of an ash colour, downy and harsh.
Leaves grow by fours, linear, obtuse, a little hairy, roughish and
spreading; with very short foot-stalks, pressed to the branches.
Flowers grow on the upper branches in loose spikes, pointing one way,
nodding and in bunches; the foot-stalks very short, having three small floral
leaves.
Empalement. Cup of four leaves, which are tapered, very short, smooth
and pressed to the blossom.
Blossom, bell-shaped, small, sweet-scented, and white; the segments
upright, with the points a little bent outwards.
Chives. Eight hair-like threads, curved at the end. Tips bearded, and
within the blossom.
Pointal. Seed-bud nearly globular and furrowed. Shaft without the
blossom. Summit four-cornered.
Native of the Cape of Good Hope.
Flowers from October, till February.

REFERENCE.

1. A Leaf, magnified.
2. The Empalement, and Blossom.
3. The Empalement, magnified.
4. The Chives, magnified.
5. The Pointal, magnified.
 ERICA calycina.

CHARACTER SPECIFICUS.

Erica, antheris cristatis, inclusis; foliis ternis; rami filiformes; corollis

minutis, urceoli-formes; calycis foliola corolla longiora; pedunculis
longissimis.

DESCRIPTIO.

Caulis filiformis, spithameus, erectus, ramosissimus.

Folia terna, lanceolata, supra concava, adpressa, glabra; petiolis
brevissimis.
Flores plures, in ultimis ramulis terminales, umbellati; pedunculis
carneis, longissimis, bracteis tribus coloratis instructis.
Calyx. Perianthium tetraphyllum, foliolis lato-ovatis, carneis, corolla
longiora.
Corolla minuta, urceoli-formis, apice dilute carnea, ad basi virescens;
oris laciniis reflexis, saturate carneis.
Stamina. Filamenta octo capillaria; antheræ cristatæ, inclusæ.
Pistillum. Germen sub-globosum. Stylus sub-exsertus filiformis. Stigma
tetragonum.
Habitat ad Caput Bonæ Spei.
Floret a mensi Septembri, in Decembrem.

REFERENTIA.

1. Calyx, et Corolla.
2. Corolla.
3. Calyx, lente auctus.
4. Stamina, et Pistillum.