Research Projects

NLU Project

Lecture Project, Institute for Machine Learning, ETH Zürich, Zurich

In a team of two:

• Implementing a simple LSTM Language model to perform various experiments.
• Implementing a system that can solve the Story Cloze task. In the Story Cloze Task we were given a four-sentence short story and two alternatives for the last, fifth sentence that ends the story. Our designed system should decide which among those two are correct. The main challenge lies in the fact that the training data consists of five-sentence short stories that include the correct ending but there is no incorrect ending provided. Due to this to use the training data, we need to either generate wrong endings or use another source for training. The validation set consists of correct and incorrect endings but it is much smaller than training dataset to train. We approached this task by using alternate reading comprehension dataset RACE and training on data generated from it.
• Tools: Python, Tensorflow

Flood-Water Estimation through Semantic Image Interpretation

Master Thesis Project, EcoVision Lab,PRS Group,ETH Zürich; Computer Vision Group, TU München, Zürich, München

We propose a method to quantify flood water from images gathered from social media. Quantifying flood height by looking just at images is a difficult task. Therefore, in this work we are using common objects of known dimension which are partially submerged in flood water to quantify flood height. There are various factors which makes this task difficult:

• images are cluttered due to presence of different classes;
• objects of different classes present in various scales depending on the distance at which the image was taken;
• detection of occluded objects as they are partially submerged in flood water. Also, high intra-class variability of instances of objects like buildings or houses and cars.

We model this problem on two levels. For the first part, we train a network to give per image floodwater-level. For second part, we train a model to give more finer classification. For every instance of an object in an image, the model will predict the class of the object and level of flood. Through this project we also contribute a dataset of flood images gleaned from social media.

Flood-Water Estimation from Social Media Images

PhD Project , EcoVision Lab, Photogrammetry and Remote Sensing, ETH Zürich, Zurich

In this work, we extended the work done on my Master thesis project. The paper on the project was published in ISPRS Annals and can be found here Paper. Our paper won the Best Paper Award at ISPRS Geospatial Week 2019 in Semantic Scene Analysis and 3D Reconstruction from Images and Images Sequences track. Award. For our future work, we also want to move from fully-supervised task to weak supervision with learning to rank module. The motivation is to use the ordinal information of flood levels which is ignored when training with regression or classification task. Also, the weak supervision required to train a ranking network is a much simpler annotation task than pixel-wise annotation of images.

Data Mining Lab

Advanced Practical Course, ,Chair of BioInformatics TU München, München

In a team of three:

• During the course, we went through the whole path of data mining from dataset preparation up to meaningful predictions.
• It included the following steps: dataset search and description, understanding the data and naive introspection, feature construction and selection and prediction and evaluation.
• We worked on the Mashable dataset from UCI repository. After finding inconsistencies in the dataset we rebuild the dataset ourselves. We did descriptive analysis and text mining on the data and prepared our final features. Our final goal was to predict the number of shares on social media that reflects an articles’s popularity and give recommendations to the author in case the prediction is unpopular.
• Tools: Python, R