Graph-based Bag-of-Words for Classification

This paper, entitled Graph-based Bag-of-Words for Classification, introduces the Bag of Graphs (BoG), a Bag-of-Words model that encodes in graphs the local structures of a digital object. It presents a formal definition, introducing concepts and rules that make this model flexible and adaptable for different applications. It is defined two BoG-based methods – Bag of Singleton Graphs (BoSG) and Bag of Visual Graphs (BoVG), which create vector representations for graphs and images, respectively. The hypothesis explored in this paper is that the combination of graphs with the BoW model can create a discriminant and efficient representation based on local structures of an object, leading to fast and accurate results in classification tasks. The rationale is that the two representations are complementary and can help each other overcome their individual deficiencies.

The authors evaluate the Bag of Singleton Graphs (BoSG) for graph classification on four datasets of the IAM repository, obtaining significant results in accuracy and execution time. The method Bag of Visual Graphs (BoVG) is evaluated for image classification on Caltech and ALOI datasets, and for remote sensing image classification on images of Monte Santo and Campinas datasets. This framework opens possibilities for retrieval, classification, and clustering tasks on large datasets that use graph-based representations impractical before due to the complexity of inexact graph matching.

Fernanda B. Silva, Rafael de O. Werneck, Siome Goldenstein, Salvatore Tabbone, Ricardo da S. Torres, Graph-based Bag-of-Words for Classification, Pattern Recognition (2017), doi: 10.1016/j.patcog.2017.09.018

Posted in blog, publications, science | Tagged , , , , , , , , , | Leave a comment

Talk: Remote Sensing Image Analysis in Phenology Studies

Tomorrow, on Oct 6th 2017 at 14h IC/Unicamp auditorium, Prof. Ricardo Torres will give a talk about Remote Sensing Image Analysis in Phenology Studies. The talk is part of the Institute of Computing weekly seminars series of 2017.

Title: Remote Sensing Image Analysis in Phenology Studies: Challenges and Research Opportunities

Abstract: Environmental changes are becoming an important issue in the global agenda. Typical change understanding ecological studies rely on the use of huge volumes of remote sensing image, demanding the definition of effective and efficient services for appropriate storing, retrieval, and knowledge extraction. This lecture will focus on presenting ongoing research initiatives focused on the specification and implementation of appropriate systems to handle large-scale remote sensing image collections. Special attention will be given to recent research results on image processing, machine learning, and time series analysis in the context of the e-Phenology project. The e-Phenology is a multidisciplinary project that combines research efforts in Computer Science and Phenology. Its objective is to address practical and theoretical problems involved in the use of new technologies to the remote observation of phenology, aiming to detect local environmental changes.

Posted in blog, Keynotes, science, talk | Tagged , , , , | Leave a comment

Two Ph.D. thesis defense: Hilario Seibel and Danille Rodrigues

The RECODists Hilario Seibel Jr. and Daniele Rodrigues have earned their Ph.D. degrees last Friday September 29th at IC/Unicamp. The details are shown below:

Ph.D. candidate: Hilario Seibel
Title: Super-resolution in low-quality videos for forensics, surveillance, and mobile applications
Supervisor: Siome Goldenstein
Co-supervisor: Anderson Rocha

Super-resolution (SR) algorithms are methods for achieving high-resolution (HR) enlargements of pixel-based images. In multi-frame super resolution, a set of low-resolution (LR) images of a scene are combined to construct an image with higher resolution. Super resolution is an inexpensive solution to overcome the limitations of image acquisition hardware systems, and can be useful in several cases in which the device cannot be upgraded or replaced, but multiple frames of the same scene can be obtained. In this work, we explore SR possibilities for natural images, in scenarios wherein we have multiple frames of a same scene. We design and develop variations of an algorithm which rely on exploring geometric properties in order to combine pixels from LR observations into an HR grid; two variations of a method that combines inpainting techniques to multi-frame super resolution; and three variations of an algorithm that uses adaptive filtering and Tikhonov regularization to solve a least-square problem.

Multi-frame super resolution is possible when there is motion and non-redundant information from LR observations. However, combining a large number of frames into a higher resolution image may not be computationally feasible by complex super-resolution techniques. The first application of the proposed methods is in consumer-grade photography with a setup in which several low-resolution images gathered by recent mobile devices can be combined to create a much higher resolution image. Such always-on low-power environment requires e active high-performance algorithms, that run fastly and with a low-memory footprint.

The second application is in Digital Forensic, with a setup in which low-quality surveil- lance cameras throughout the cities could provide important cues to identify a suspect vehicle, for example, in a crime scene. However, license-plate recognition is especially di cult under poor image resolutions. Hence, we design and develop a novel, free and open-source framework underpinned by SR and Automatic License-Plate Recognition (ALPR) techniques to identify license-plate characters in low-quality real-world traffic videos, captured by cameras not designed for the ALPR task, aiding forensic analysts in understanding an event of interest. The framework handles the necessary conditions to identify a target license plate, using a novel methodology to locate, track, align, su- per resolve, and recognize its alphanumerics. The user receives as outputs the rectitude and super-resolved license-plate, richer in details, and also the sequence of license-plates characters that have been automatically recognized in the super-resolved image.

We present quantitative and qualitative validations of the proposed algorithms and its applications. Our experiments show, for example, that SR can increase the number of correctly recognized characters posing the framework as an important step toward providing forensic experts and practitioners with a solution for the license-plate recognition problem under difficult acquisition conditions. Finally, we also suggest a minimum number of images to use as input in each application.

This slideshow requires JavaScript.

Ph.D. candidate: Daniele Rodrigues
Title: Complex Network Measurements in Graph-based Spatio-Temporal Soccer Match Analysis
Supervisor: Ricardo Torres

Soccer match analysis is of paramount importance in the definition of appropriate training programs and game strategies. The increasing availability of sport-related data in the recent years, due to the use of modern tracking systems, has allowed advances in sports analytics, providing coaches with valuable information for match and teams analysis. The availability of these data, on the other hand, challenges science to develop tools capable of storing, visualizing, and analyzing this large volume of information. Soccer analyses are usually performed using matches’ statistics, events (e.g., passes and shots on goal) and players location data. Related studies have been representing the matches’ events as a single graph, where players are vertices and edges are actions performed among them during the match. The graph is then analyzed from a complex network measurement perspective. Although this approach provides interesting insights about the tactical actions occurred during the game, revealing some tactical patterns, it disregards the spatio-temporal aspects inherent to the sport, as the positioning of the players on the pitch, and the moment in time when relevant actions occur.

This thesis addresses these shortcomings by presenting a soccer game analysis framework. We propose a new approach for soccer match analysis, based on graphs, that considers the spatio-temporal characteristics, intrinsic to the dynamic of soccer. We propose to represent the match as a temporal graph, by encoding players’ location on the pitch into instant graphs, in which vertices represent players in their real location and edges are defined based on their distance in the field and the possibility of short pass exchanges. We demonstrate that this representation, named opponent-aware graph, which takes into account the presence of opponents, and the diversity entropy measurement are effective tools for determining the role of attacking players in a match and the probability of successful passes. By taking into account different measurements of complex networks in temporal graphs, this study also investigates the feasibility of using complex network measurements and machine learning algorithms to characterize the role of players in a match. The results allow to further characterize the decision-making process of players, providing interesting insights to coaches and researchers for possibly improving training strategies. This study also addresses the visualization of temporal graphs problem by introducing the Graph Visual Rhythm, a novel image-based representation to visualize changing patterns typically found in temporal graphs. This representation is based on the concept of visual rhythms, motivated by its capacity of providing a lot of contextual information about graph dynamics in a compact way. We validate the use of graph visual rhythms through the creation of a visual analytics tool to support the decision-making process based on complex-network-oriented soccer match analysis.

This slideshow requires JavaScript.

Posted in blog, thesis defense | Tagged , , , , , , | Leave a comment

FAPESP Thematic Projects at RECOD

RECOD Lab. has been awarded two Fapesp Thematic Projects recently. This is an AMAZING achievement for our lab. and for IC/Unicamp, especially during this tough financial moment in our country. The two projects, entitled “DéjàVu: Feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events” — coordinated by Prof. Anderson Rocha — and “The secret of playing football: Brazil versus the Netherlands” — coordinated by Prof. Ricardo Torres are starting this semester. Details about each project are given below.

DéjàVu: Feature-Space-Time Coherence from Heterogeneous Data for Media Integrity Analytics and Interpretation of Events

FAPESP Thematic Project 2017/12646-3,  from December 2017 through November 2022

In this research project, we focus on synchronizing specific events in space and time (X-coherence), fact-checking, and mining persons, objects and contents of interest from various and heterogeneous sources including — but not limited to — the internet, social media and surveillance imagery. For that, we seek to harness information from various media sources and synchronize the multiple textual and visual information pieces around the position of an event or object as well as order them so as to allow a better understanding about what happened before, during, and shortly after the event. After automatically organizing the data and understanding the order of the facts, we can devise and deploy solutions for mining persons or objects of interest for suspect analysis/tracking, fact-checking, or even understanding the nature of the said event. Additionally, by exploring the possible existing links among different pieces of information, we aim at further designing and developing media integrity analytics tools to hint at existing forgeries, sensitive content (e.g., violent content, child pornography), and spreading patterns of multimedia objects online. With demanding and sophisticated crimes and terrorist threats becoming ever more pervasive, allied with the advent and spread of fake news, our objective is to use the developed solutions to help us answering the four most important questions in forensics regarding an event: “who”, “in what circumstances”, “why”, and “how”, thus identifying the characteristics and circumstances in which an event has taken place.

Hélio Pedrini, Co-PI (Unicamp), André C.P.L.F. de Carvalho, Co-PI (USP), Adam Czajka (Warsaw Univ. of Technology, Poland), Alex Kot (Nanyang Technological University, Singapore), C.-C. Jay Kuo (Univ. of Southern California, USA), Edward Delp (Purdue Univ., USA), Kevin Bowyer (Univ. of Notre Dame, USA), Patrick Flynn, (Univ. of Notre Dame, USA), Stefano Tubaro (Politecnico di Milano, Italy), Paolo Bestagini (Politecnico di Milano, Italy), Walter J. Scheirer (Univ. of Notre Dame, USA), Alexandre Ferreira (UNICAMP), Adin Ramirez Rivera(UNICAMP), Eduardo Valle, (UNICAMP), Fernanda Alcântara Andaló (UNICAMP), Jacques Wainer (UNICAMP), Lin Tzy Li (UNICAMP), Sandra Avila (UNICAMP), Zanoni Dias (UNICAMP), Marcos André Gonçalves (UFMG), William Schwartz (UFMG).

The Secret of Playing Football: Brazil versus the Netherlands

FAPESP Thematic Project 2016/50250-1 from August 2017 through July 2021

The scientific challenge is about unraveling the secret of Brazilian and Dutch soccer by capturing successful elements of game play of both countries, combining expertise from data science, computer science, and sport science. Suggested features from literature, as well as several novel ones, will be considered and filtered on how they capture success in soccer. A manageable set of features will then be obtained from various available Dutch datasets (focusing on the successful play). Subsequently, the same features will be used to compare playing styles between both countries. Features of game play will be approached from two different angles. The first angle (spearheaded by the Brazilian computer science partner) concerns features that capture the dynamics of game play and characterize aspects of formation on the pitch. The second angle (lead by the Dutch data science partner) will focus on how an attack is built up, and how key events (shots on goal, transitions from defenders to midfielders, etc.) can help to characterize this. For the comparison between countries data will be collected in four different age categories in Brazil and the Netherlands during official games, in order to compare (the development of) game play between both countries. Data will be collected by means of the Local Position Measurement System, for reasons of accuracy and consistency. The applied science part of this proposal is focusing on bridging the gap between fundamental science and soccer practice, i.e. coaches, trainers, clubs, and federations. The outcomes of the fundamental part will be implemented in a coach-cockpit, a software application which trainers and coaches can use to (1) decide upon their strategy before a game, (2) analyze player and team behaviour during a game enabling to adjust the strategy accordingly, and (3) choose and/or design training forms to improve player and team behaviour.

Claudio Alexandre Gobatto (UNICAMP), Sergio Augusto Cunha (UNICAMP), Ricardo Oliveira Anido (UNICAMP), Luiz Eduardo Barreto Martins (UNICAMP), Ricardo Machado Leite de Barros (UNICAMP), Felipe Arruda Moura (UEL) e Paulo Roberto Pereira Santiago (USP), Koen A. P. M. Lemmink (University of Groningen), Johan Pion (HAN University), Marije T. Elferink-Gemser (University of Groningen), Joost N. Kok (Leiden University), e Arno Knobbe (Leiden University).

Posted in awards, blog, science | Tagged , , , , , , | Leave a comment

Kuaa: A unified framework for design, deployment, execution, and recommendation of ML experiments

Several libraries and machine-learning frameworks have been proposed in the literature to support users in the process of defining the most appropriate methods for their applications. However, many frameworks have limitations including the lack of flexibility to include novel proposed descriptors and machine learning methods, and specially, the inability to reuse previous experiments and learn from them.

In this article published in Future Generation Computer Systems, the authors propose Kuaa, a workflow-based framework that can be used for designing, deploying, and executing machine learning experiments in an automated fashion. This framework is able to provide a standardized environment for exploratory analysis of machine learning solutions, as it supports the evaluation of feature descriptors, normalizers, classifiers, and fusion approaches in a wide range of tasks involving machine learning. Kuaa also is capable of providing users with the recommendation of machine-learning workflows. The use of recommendations allows users to identify, evaluate, and possibly reuse previously defined successful solutions. The authors propose the use of similarity measures (e.g., Jaccard, Sørensen, and Jaro–Winkler) and learning-to-rank methods (LRAR) in the implementation of the recommendation service.

Experimental results show that Jaro–Winkler yields the highest effectiveness performance with comparable results to those observed for LRAR, presenting the best alternative machine learning experiments to the user. In both cases, the recommendations performed are very promising and the developed framework might help users in different daily exploratory machine learning tasks.

Kuaa Code:

Rafael de Oliveira Werneck, Waldir Rodrigues de Almeida, Bernardo Vecchia Stein, Daniel Vatanabe Pazinato, Pedro Ribeiro Mendes Júnior, Otávio Augusto Bizetto Penatti, Anderson Rocha, Ricardo da Silva Torres, Kuaa: A unified framework for design, deployment, execution, and recommendation of machine learning experiments, Future Generation Computer Systems, Volume 78, 2018, Pages 59-76, ISSN 0167-739X,

Posted in blog, publications, science | Tagged , , , , , , , , , , , , | Leave a comment

RECOD on the news

An article published at Correio Braziliense (in Portuguese), which is an important vehicle of communication in Brazil, highlights that 40% of doctored images are not perceived by people. This conclusion was reached by researchers at Warwick University through several experiments. Prof. Anderson Rocha was invited to comment this scenario and to talk about our efforts in developing algorithms capable of identifying these manipulations in an automatic fashion. The article discuss the importance of an automatic image manipulation tool and its impact in our modern digital society.

O que os olhos não veem (in Portuguese)

Posted in blog, extra, media | Tagged , , , , | Leave a comment

RECOD amasses (for the second time) two awards at the Google Research Awards for Latin America 2017

As a good déjà vu of what happened in 2016, two RECOD projects coordinated by Prof. Anderson Rocha and Prof. Eduardo Valle will keep receiving their scholarships from Google LARA (Latin American Research Awards) for the next 12 months.

This year there were 281 projects submitted but only 27 awarded. The result reaffirms the serious research work we have been doing at RECOD in the Computer Science field and, in particular, in Machine Learning.

The projects are:

Prof. Anderson Rocha and PhD candidate José Ramon Trindade Pires

Automated Data­-Driven Screening of Diabetic Retinopathy – Extension

In this research, we aim to design methods to recognize discriminative patterns of diabetic retinopathy stages, providing an advanced and robust severity decision; and incorporate such information into a final higher-level (and more refined) decision of referable DR. We also intend to explore possible forms of understanding the decisions taken by the devised solutions toward accountable decision-making methods.

Prof. Eduardo Alves do Valle Junior and PhD candidate Michel Fornaciali

Reliable Automated Melanoma Screening for the Real World

This project aims to speed-up real-world adoption of computer-aided melanoma screening, both by enhancing the Machine Learning models used to detect the disease, and by interacting with doctors to identify and remove barriers that prevent the adoption of the technology.

The event was covered by our local press (in Portuguese):


This slideshow requires JavaScript.

Posted in awards, blog | Tagged , , , , , , , , | Leave a comment

Your PLOS ONE Article is in the Top 25% Most Cited

The research paper entitled “Advancing Bag-of-Visual-Words Representations for Lesion Classification in Retinal Images” is among the top 25% most cited articles at PLOS ONE journal.

This article inaugurates RECOD’s policy on improving the reproducibility of our published results, by making publicly available both data and code as much as allowable by copyright law and by ethical practice.


Posted in awards, blog, media | Tagged , , , , | Leave a comment

User-Centric Coordinates for Applications Leveraging 3-Axis Accelerometer Data

Mobile devices are becoming ubiquitous and, sometimes, even extensions of ourselves. These devices are growing fast in terms of delivered computational power, storage capacity, battery duration, and built-in sensors. Time and again, we see headlines advertising new unforeseen applications leveraging this power, especially the sensors, for solving diverse problems, including fall detection, user’s activity recognition, location identification, or even user authentication based on the way of walking (gait).

In this paper, the authors focus on motion sensors and discuss how the provided data can be interpreted and transformed to better serve different purposes. They propose a method to process the data from such sensors that reduces the acquisition noise and possible artifacts, and turns the data invariant to the device’s position and the user’s movement direction. A new coordinate system referred to as user-centric is introduced, as opposed to the two most common coordinate systems used—the device and world-coordinate systems. The results show the importance of properly pre-processing the acquired data to enable more reliable applications underpinned by mobile sensors.

The figure shows the acceleration data represented in the world-coordinate system (left side) and in the user-centric coordinate system (right side). The samples are divided into three straight lines, represented by A, B, and C slots, and two left turns. In the world-coordinate system, after the first user’s left turn, we can notice the acceleration data from axis x in slot A moving to axis y in slot B, as blue box shows. This phenomenon happens again after the second user’s left turn, from slot B to slot C. It indicates that the direction the user is following is somehow represented in the acceleration data. Because it aims at being as much independent as possible from external factors – such as the user’s path – the coordinate systems shall not capture such variations. This is exactly what happens in the proposed user-centric coordinate system, in which the acceleration samples are kept in the same axis even after the user’s left turns.

Ferreira, A., Santos, G., Rocha, A., & Goldenstein, S. (2017). User-Centric Coordinates for Applications Leveraging 3-Axis Accelerometer Data. IEEE Sensors Journal, 17(16), 5231-5243.

Posted in blog, publications, science | Tagged , , , , , , , , , | Leave a comment

RECOD on the news

With the advances of artificial intelligence (AI), specially machine learning, many areas can take advantages of these emerging technologies. The learning experience, for instance, could be much more personal with algorithms that are able to identify possible difficulties of one particular student regarding specific topics and thus provide additional material or extra exercises.

In the following interviews, Prof. Anderson Rocha from RECOD has highlighted some important points in the relationship between AI and the modern society.

1) Pais do milênio confiam na inteligência artificial para educação dos filhos (O Globo, 20 jul. 2017 – in Portuguese)

2) Pais jovens confiam na inteligência artificial para educação dos filhos (Pequenas Empresas & Grandes Negócios, 20 jul. 2017 – in Portuguese)

Posted in blog, extra, media | Tagged , , , , , | Leave a comment