CoviHawkes: An AI tool for Covid-19

In the last one and a half years, more than 17 Crore cases and 37 Lakh deaths have been reported worldwide due to Covid-19. This has been a devastating time for everyone. Recently, the sudden rise of cases in India, the second wave of infections, prompted strict lockdowns across the country. While these lockdowns effectively curb the spread of the virus, they often come with severe emotional and financial consequences that tend to outlast the lockdown itself. Science-based and data-driven policies for enacting lockdowns at local levels are the need of the hour. Our tool can help policymakers identify local regions that must go into lockdown mode to control the infection rate. Arguably, such local lockdowns would offer a sounder alternative to state-wide or nationwide lockdowns.

This tool is powered by powerful machine learning techniques. We collect data at three levels: country, state, and districts. This data is used to forecast the daily case counts at each of these levels in the future. Our tool combines the observed patterns in the case counts from the past with additional factors like demographics (e.g., population density) and mobility (e.g., decrease in the percentage of people traveling for work due to lockdown) of the region. These forecasts can be used by policymakers to strategically enact lockdowns in those places that are deemed vulnerable in the near future.

This tool is available at the following link.

https://sml.csa.iisc.ac.in/covihawkes/

A Vapnik’s Imperative to Unsupervised Domain Adaptation

The success of deep neural networks, in the advancement of AI, can be attributed to models trained on large amounts of labeled data. Acquiring massive amounts of labeled data in some domains can be very expensive or not possible at all. To address this one can recourse to cross-domain learning. In our work, by following Vapnik’s imperative of statistical learning, we propose a direct approach to domain adaptation called Contradistinguisher. With this technique, we set benchmark results in almost all available datasets that are studied in the literature.

References:

S. Balgi and A. Dukkipati. Contradistinguisher: A Vapnik’s Imperative to Unsupervised Domain Adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence: 2021

Website: https://www.csa.iisc.ac.in/~ambedkar/

Advanced Healthcare Technologies

The research group at BEES Lab, DESE, is developing advanced healthcare technologies to screen and diagnose breast, oral, and brain cancer. The group develops single and multi-shank silicon microneedles integrated with microelectrodes and related signal conditioning modules to record neural biopotentials and signals and electrical stimulation of deep brain regions for understanding stroke, the efficacy of anti-epileptic drugs, and therapeutic interventions for Parkinson’s disease. To record the brain’s surface biopotentials and sub-dural signals, such as the electrocorticogram (ECoG), flexible devices with biocompatible materials with recording electrodes at microscale resolution are designed and fabricated. The focus is also on developing smart steerable catheters for managing airway obstruction in infants and ablation catheters.  The group collaborates with twenty-two clinicians towards translational research.

References:

Uttam M. Pal, Anil Vishnu GK, Saeed Rila, Saahil Shroff, Gokul A M, Deb Baruah, Jayant S. Vaidya, Gayatri Gogoi, and Hardik J. Pandya, “Portable near-infrared spectroscopy tool using multi-spectral non-contact probes to delineate normal and cancerous breast tissue.” IEEE TBioCAS, 14(4), 879-888, 2020.  https://doi.org/10.1109/TBCAS.2020.3005971

B Alekya, B S Arjun, V Bhushan, Kevin Abishek, Sanjay Rao, Yeongjin Kim, and   Hardik J. Pandya, “An intubation catheter integrated with flow sensors and smart actuators for characterizing airflow patterns in stenosed trachea: an objective guide for CAO management”, Journal of Micromechanics and Microengineering, 31 (5), 055007, 2021.

Uttam Pal, Ashika Nayak, Tejaswi Medisetti, Gayatri Gogoi, Himanshu Shekhar, Msn Prasad, Jayant S Vaidya, Hardik J. Pandya, “Hybrid Spectral-IRDx: Near-IR and Ultrasound Attenuation System for Differentiating Breast Cancer from Adjacent Normal Tissue”, IEEE TBME, 2021. (online ahead of print) https://doi.org/10.1109/TBME.2021.3077582

Inventions:

Invention Title: An in-vivo, intraoperative probe for brain tumor margin delineation and methods thereof
Inventors: Hardik J. Pandya, Arjun B S, Anil Vishnu G K, Gokul A M, Arun Baby, Shilpa Rao, Manish Beniwal, Vikas V, Anita Mahadevan
Indian patent application filing no: 202041022728
Date of Filing: May 30, 2020

Invention Title: Apparatus for high-throughput rapid antibiotic susceptibility testing and methods thereof
Inventors: Hardik J. Pandya, Anil Vishnu G. K., Bhagaban Behera, Alekya B., Arun Baby, Saeed Rila, Arjun B. S., Midhun C. Kachappilly, Prathik B.H., Nagasuma Chandra, Dipshikha Chakravortty
Indian patent application filing no: 202041024394
Date of Filing: June 10, 2020

Invention Title: A handheld diagnostic tool for grading stenosis in pediatric upper airway and methods for characterizing the same
Inventors: Hardik J. Pandya, Alekya B, V S N Sitaramgupta, Arjun B. S Bhushan V, S Siddesh Shenoy, Sanjay Rao, Mayur Bhuva, Kevin Abhishek
Indian patent application filing no: 202041027223
Date of Filing: June 26, 2020

Invention Title: Apparatus and System for Rapid diagnosis of Malignant Lesions from Biopsy Tissues for use Inside the Operation Room
Inventors: Hardik J. Pandya, Anil Vishnu G. K., Bhagaban Behera, Saeed Rila, Midhun C. Kachappilly, Arun Baby, Annapoorni Rangarajan
Indian patent application filing no: 202041029955
Date of Filing: July 14, 2020

Website: https://eecs.iisc.ac.in/people/hardik-j-pandya/

Optimal Algorithms for Statistical Inference and Learning under Information Constraints

Classical statistics assumes that data samples are available in their entirety at one place. But what happens when only limited information is allowed about each sample from a user? The information can be constrained in form of communication constraints as in federated learning or local privacy constraints. We have developed a general framework for design and analysis of statistical and optimization procedures under such distributed information constraints.

References:

J Acharya, C Canonne, and H Tyagi, “Inference under information constraints: Lower Bounds from Chi-Square Contraction,” Proceedings of the Thirty-Second Conference on Learning Theory (COLT), PMLR 99:3-17, 2019.

P Mayekar and H Tyagi, “RATQ: A Universal Fixed-Length Quantizer for Stochastic Optimization” Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 108:1399-1409, 2020.

Website: https://ece.iisc.ac.in/~htyagi/

Uniqueness of rigid point-cloud registration

A necessary and sufficient condition for uniquely registering a system of point clouds (whose positions are known in different coordinate systems) in a global coordinate system [1] has been obtained. This was done by relating the registration problem to the rigidity of bar-and-joint frameworks. In particular,  a particularly simple connectivity-based test for point clouds in two dimensions. Was obtained. Testing uniqueness however remains open in three and higher dimensions.

References:

A. V. Singh and K. N. Chaudhury, “On uniquely registrable networks,” IEEE Transactions on Network Science and Engineering, 2019.

Website: http://www.ee.iisc.ac.in/faculty/kunal/

Semi-Supervised Cross-Modal Retrieval With Label Prediction

Cross-modal retrieval with image-text, sketch-image, etc. are gaining increasing importance due to abundance of data from multiple modalities, and applications like e-commerce, security, etc. In [1], we develop a novel approach which utilizes few labeled and remaining unlabeled data for this task, thus reducing manual intervention significantly. In [2], we address the problem of retrieving previously unseen data [2], since new categories are discovered dynamically in real-world.

References:

[1] D. Mandal, P. Rao, S. Biswas. Semi-Supervised Cross-Modal Retrieval With Label Prediction, IEEE Transactions on Multimedia (TMM), September, 2020.

[2] T. Dutta, A. Singh, S. Biswas. Adaptive Margin Diversity Regularizer for handling Data Imbalance in Zero-Shot SBIR, European Conference on Computer Vision (ECCV), August, 2020.

Website: http://www.ee.iisc.ac.in/new/people/faculty/soma.biswas/index.html

Deep Unsupervised Speech Representation Learning

The performance of speech systems is degraded in the presence of noise. The principle of modulation filtering attempts to remove the spectro-temporal modulations that are susceptible to noise. While traditional approaches use modulation filters that are hand-crafted, we propose a novel method for modulation filter learning using deep variational models. Specifically, we pose the filter learning problem in an unsupervised generative modeling framework where the convolutional filters capture the modulations that are shown to improve speech recognition performance significantly.

References

Purvi Agrawal, Sriram Ganapathy:
Modulation Filter Learning Using Deep Variational Networks for Robust Speech Recognition. IEEE J. Sel. Top. Signal Process. 13(2): 244-253 (2019)

Website: http://www.leap.ee.iisc.ac.in/sriram/

Can a Machine Learn Charge Density Waves?

Charge density wave (CDW) is a quantum mechanical phenomenon, which induces distortion in the crystal structures of some low-dimensional (1D or 2D) metals, when the temperature is reduced. Such distorted crystal structure is known as CDW phase and its resistivity is much higher than the original symmetric phase. Since the switching between symmetric and CDW phase can also be made by the application of external electric field, these materials are technologically important. They find applications in memristive based in-memory computing, transistor-less logic circuit and oscillatory neural networks.

The rapid computational screening for exotic and application-specific properties in 2D materials (e.g. non-trivial topological order, high-temperature ferromagnetism, superior catalytic activity etc.) is becoming increasingly important these days. First-principle based rapid prediction of CDW phases in 2D materials has not yet been reported. Currently, there is a gaping scarcity of suitable 2D CDW materials for Nanoelectronic device applications. We bridge this gap by developing an automated high-throughput computational tool combining first-principles-based structure-searching technique and unsupervised machine learning, which identifies CDW phases from a unit cell with inherited Kohn anomaly.

Although machine learning techniques are now being extensively used in material sciences, these studies mostly involve supervised regression to predict certain material properties. Here we deploy unsupervised classification, thanks to which, we find a host of undiscovered phases for even extensively studied 2D CDW materials such as 1T-TaS2 and 2H-NbSe2. The proposed methodology not only rediscovers the known CDW phases but also predicts a host of easily exfoliable new CDW materials (a total 30 materials and 114 phases) along with associated electronic structures. Among many promising candidates, we pay special attention to ZrTiSe4 and conduct a comprehensive analysis to gain insight into the quantum-mechanical-phenomana called ‘Fermi-surface-nesting’, which causes significant semiconducting gap opening in its CDW phase.

Our findings could provide useful guidelines for the experimentalists and foster practical application of CDW materials in nanoelectronics.

References

Arnab Kabiraj and Santanu Mahapatra, “Machine-Intelligence-Driven High-Throughput Prediction of 2D Charge Density Wave Phases”, The Journal of Physical Chemistry Letters, 2020.

https://pubs.acs.org/doi/10.1021/acs.jpclett.0c01846

Presentation Video: https://youtu.be/ZGWJH5qMSr8

Website: https://faculty.dese.iisc.ac.in/santanu/

Neuromorphic CMOS-MoS2 based hybrid system for low power edge-computing

The brain is an ideal template for next-generation computing architectures. We, at NeuRonICS lab IISc, have developed a hybrid architecture [1] that combines a silicon neuron, acting as the computing unit, with a 2D nanosheet of molybdenum sulphate, acting as a memristor that works like a synapse [2]. This can improve power use and performance in IoT-based edge devices.

References:

Gupta, S., Kumar, P., Paul, T., van Schaik, A., Ghosh, A., & Thakur, C. S. (2019). Low Power, CMOS-MoS 2 Memtransistor based Neuromorphic Hybrid Architecture for Wake-Up Systems. Nature Scientific Reports, 9(1), 1-9.

Paul, T., Ahmed, T., Tiwari, K. K., Thakur, C. S., & Ghosh, A. (2019). A high-performance MoS2 synaptic device with floating gate engineering for neuromorphic computing. 2D Materials, 6(4), 045008

Teaching A GAN what not to learn

Semi-supervised generative adversarial networks (GANs) consist of neural networks tasked with learning the underlying distributions of each class of a target dataset, usually images. We approach the GAN learning problem from a novel perspective, one that is motivated by the famous Persian poet Rumi, who said, “The art of knowing is knowing what to ignore,” we provide the GAN with both positive data that it must learn to model, and negative samples that it must learn to avoid. The proposed formulation is able to generate realistic high-resolution images on datasets with positive to negative class ratios as low as 1:50.

References:

Asokan, S. and Seelamantula, C. S. “Teaching a GAN what not to Learn.” In Advances in Neural Information Processing Systems (NeurIPS), 2020

https://tinyurl.com/NeurIPSRumiGAN

https://github.com/DarthSid95/RumiGANs

https://papers.nips.cc/paper/2020/file/29405e2a4c22866a205f557559c7fa4b-Paper.pdf

AI-driven XraySetu for early-Covid interventions over WhatsApp

The project was started by Dr. Geetha Manjunath of Niramai Health and Prof. Chiranjib Bhattacharyya of IISc, in the first wave of Pandemic to address the issue of Accessibility of Healthcare in the context of COVID-19.  The key premise behind the project was X-rays are far more ubiquitous than RTPCR tests and a diagnostic tool based on X-ray images could be an important proxy for RTPCR. To ensure accessibility a tool was developed on Whatsapp platform  for easy communication with practicing Doctors . 

The technical challenge was to develop predictive models for understanding Whatsapp based X-ray images which are not only accurate but also provide some explanation with very few  instances of COVID-19 positive cases. The  model is based on  an unique Transfer Learning framework that leverages easily available X-ray images of lungs, not necessarily COVID positive, to learn useful features which have high predictive power. A  confidence score, guided by the infected areas of lungs, along with automatic annotation provides some insights into the final prediction. The system outputs a prediction, localizes the infected parts, and creates a report which gives a confidence score, all within a few minutes.  

The system was piloted in Karnataka  as COVID SWIFT (  link  for description of the working system) over the last 10 months.   Buoyed by the success of the pilot, in collaboration with Mr. Umakant Soni of ARTPARK, the tool have now been launched as X-ray Setu, a free-to-use AI-driven platform to aid doctors for early-Covid interventions over WhatsApp.  For more information see Xray Setu Website.

https://health.economictimes.indiatimes.com/news/medical-devices/ai-driven-xraysetu-for-early-covid-interventions-over-whatsapp/83033394

Reward Biased Maximum Likelihood Estimate Approach to Online Machine Learning

We revisit the Reward Biased Maximum Likelihood Estimate (RBMLE) algorithm that was proposed in (Kumar and Becker, 1982) to solve the problem of adaptive control (Richard Bellman 1961, Feldbaum, 1960a). It solved the “exploration vs. exploitation problem” by employing a bias in favor of parameters with larger optimal rewards, thereby introducing the principle of optimism in the face of uncertainty.

Modern attention is focused on the much finer notion of “learning regret” (Lai and Robbins, 1985). We show that RBMLE has a regret of O(logT) over a time horizon of T steps, similar to state-of-the-art algorithms. Simulation studies show that RBMLE outperforms other algorithms such as UCRL2 and Thompson Sampling.

References:

Akshay Mete, Rahul Singh and P. R. Kumar, “Reward Biased Maximum Likelihood Estimation for Reinforcement Learning”, Learning for Dynamics & Control, 2021

Estimating the COVID-19 burden in Karnataka

This work, a Karnataka-wide COVID-19 serological survey, provided a wealth of information about the state of the pandemic. It enabled principled and data-driven public health responses (testing strategies, district-level response recommendations, etc.). It highlights how to leverage the existing public health infrastructure, gather useful local information, and shape targeted responses at the district/taluk level. It suggests a replicable and optimised sampling frame for enhanced information gathering in the middle of a pandemic. The four-state disease modelling framework and a combined active- and past-infection estimation technique highlight the importance of disease-state modelling in analysing complex and incomplete field epidemiological data.

References:

Giridhara R. Babu, et al., The burden of active infection and anti-SARS-CoV-2 IgG antibodies in the general population: Results from a statewide survey in Karnataka, India, International Journal of Infectious Diseases, Volume 108, P27-36, July 01, 2021.

https://cni.iisc.ac.in/covid-19-response/

No reference video quality assessment

We consider the problem of robust no reference (NR) video quality assessment (VQA) where the algorithms need to have good generalization performance when they are trained and tested on different datasets. We specifically address this question in the context of predicting video quality for compression and transmission applications. Motivated by the success of the spatio-temporal entropic differences video quality predictor in this context, we design a framework using convolutional neural networks to predict spatial and temporal entropic differences without the need for a reference or human opinion score. This approach enables our model to capture both spatial and temporal distortions effectively and allows for robust generalization. We evaluate our algorithms on a variety of datasets and show superior cross database performance when compared to state of the art NR VQA algorithms.

References:

S. Mitra, R. Soundararajan and S. S. Channappayya, “Predicting Spatio-Temporal Entropic Differences for Robust No Reference Video Quality Assessment,” in IEEE Signal Processing Letters, vol. 28, pp. 170-174, 2021,

Ballooning Multi-Armed Bandits

Online platforms such as Q&A forums (e.g. StackExchange, Reddit) receive high volume content that streams in from different sources at high speed. These platforms need to rapidly identify and display high quality content  to the users after weeding out garbage content.  We formulate this problem using  Multi-Armed Bandits (MAB) with arms modeling arriving content. We  call the model  “Ballooning MAB” since the arms (streaming inputs) grow with time.  The objective is to achieve extremely low, sublinear regret in being able to display the highest quality content by observing the user reactions to the content.  First, we show that when  new content arrives with unknown, uniformly distributed  quality, it is impossible to achieve a sublinear regret guarantee.    We then make a realistic assumption, motivated by several empirical studies in the literature, that the best (highest quality) arms are expected to arrive “early.”  This enables to  drop “later” arms without exploring them at all and we design a threshold-based algorithm that achieves sublinear regret. In particular, we exactly quantify the sublinear regret when the best arm arrival follows a sub-exponential or sub-Pareto tail distribution and show that the algorithms achieve substantially low sublinear regret.

This is joint work with Ganesh Ghalme (Postdoc, Technion); Swapnil Dhamal (Postdoc, Chalmers); Shweta Jain (IIT-Ropar); and Sujit Gujar (IIIT-Hyderabad).

References:

Ganesh Ghalme, Swapnil Dhamal, Shweta Jain, Sujit Gujar, Y. Narahari. Ballooning Multi-Armed Bandits, Artificial Intelligence, Volume 296, 2021, 31 pages.,

Machine Learning with Comparisons

Together with graduate student Aadirupa Saha at IISc, Aditya Gopalan’s work on online learning with relative preferences has resulted in a new framework for optimizing over sets of alternatives with only relative, subset-wise observations. This is a general framework that is applicable to many automated recommendation systems that can sequentially elicit only  relative preferences from, say, human users, e.g., “Do you like X over Y (and Z)?” This study has yielded state-of-the-art learning algorithms that make optimal subset selection decisions in terms of regret and rank-order estimation error, along with new insights on how to efficiently make comparisons to elicit items’ utilities within a wide range of social choice models.

References:

Aadirupa Saha and Aditya Gopalan, From PAC to Instance-Optimal Sample Complexity in the Plackett-Luce Model. 37th International Conference on Machine Learning (ICML), 2020.

Aadirupa Saha and Aditya Gopalan, Best-item Learning in Random Utility Models with Subset Choices. 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.

Multiclass- SVM Relay for Microgrid Protection

Our approach to microgrid protection in this paper uses a classification approach using multi-class Support Vector Machine (SVM) for improving the accuracy in relay coordination. Without the need for using any extensive communication facility, the proposed approach utilizes  local information to identify the corresponding SVM class to coordinate the relays to maintain the Coordination Time Interval (CTI) within the desired limits. The SVM relay has been implemented on 200-MHz TMS320F28377S, 32-bit floating point microcontroller and tested on a 4-bus distribution system. A prototype of the proposed relay has been tested on a 400V 4-Bus laboratory type microgrid system by creating different types of faults. The results obtained show that the proposed relay outperforms conventional relaying schemes reported in the literature.

References:

Vinod V and U J Shenoy, “An adaptive relay based on curve fitting technique for micro-grid protection,” IEEE International conference on Power Electronics, Smart Grid and Renewable Energy (PESGRE 2020), January 2-4, 2020 , Cochin, India.

Vinod V and U J Shenoy, “Implementation of Support Vector Machine Based Relay Coordination Scheme for Distribution System with Renewables,” IEEE Journal of Emerging and Selected Topics in Industrial Electronics, April 2021, DOI: 10.1109/JESTIE.2020.3014875.

Robust Quadrupedal Locomotion

In this work, with a view toward fast deployment of locomotion gaits in low-cost hardware, we use a linear policy for realizing end-foot trajectories in the quadruped robot, Stoch2. In particular, the parameters of the end-foot trajectories are shaped via a linear feedback policy that takes the torso orientation and the terrain slope as inputs. The corresponding desired joint angles are obtained via an inverse kinematics solver, and tracked via a PID control law. Augmented Random Search, a model-free and a gradient-free learning algorithm, is used to train this linear policy. Simulation results show that the resulting walking is robust to external pushes and terrain slope variations. This methodology is not only computationally light-weight, but also uses minimal sensing and actuation capabilities in the robot, thereby justifying the approach.

This work is in collaboration with Dr. Bharadwaj Amrutur, Dr. Shalabh Bhatnagar and Dr. Ashitava Ghosal from IISc.

Reference:

Kartik Paigwar, Lokesh Krishna, Sashank Tirumala, Naman Khetan, Aditya Sagi, Ashish Joglekar, Shalabh Bhatnagar, Ashitava Ghosal, Bharadwaj Amrutur, and Shishir Kolathaya, “Robust Quadrupedal Locomotion on Sloped Terrains: A Linear Policy Approach” 4th Conference on Robot Learning (CoRL 2020), Cambridge MA, USA.

https://stochlab.github.io/img/CORL.gif

Unsupervised Deep Architectures for Attributed Network Embedding

Outliers can have an adverse effect on the embeddings of the other nodes in a network. This happens because the outlier nodes drive the random walks across communities and hence homophily property is violated in the resulting embeddings. To overcome this problem, the effect of outlier nodes in the overall embedding objective needs to be minimized while generating the embeddings of nodes. we proposed an unsupervised algorithm which reduces the effect of outliers in network embedding by using a matrix factorization approach. Further, to accommodate possible nonlinear representations, we proposed two unsupervised deep models.

Reference:

Sambaran Bandyopadhyay, Lokesh N, Saley Vishal Vivek, M. Narasimha Murty: “Outlier Resistant Unsupervised Deep Architectures for Attributed Network Embedding”, WSDM 2020: 25-33

Sambaran Bandyopadhyay, Saley Vishal Vivek, M. Narasimha Murty: “Integrating Network Embedding and Community Outlier Detection via Multiclass Graph Description”, ECAI 2020: 976-983

Instructing Robots through gestures, text and speech

We are exploring the use of gestures and natural language (text to begin with and eventually speech), to instruct robots to do complex tasks. Not only do we want to tell robots what to do, we would like to also tell it how to do it as well as sometimes say what it should not do. More details of our current work is available in the two references below.

References:

Spatial reasoning from Natural Language Instructions for Robot Manipulation, Sagar et. al., ICRA 2021

Multi-Instance Aware Localization for End-to-End Imitation Learning, Sagar G.V., Raviteja Upadrashta, Shishir Kolathaya, Bharadwaj Amrutur, IROS 2020.

AI/ML for Next Generation Communication Systems

We are investigating the use of AI/ML techniques for communications related applications. An immediate use case seems to be to use these techniques to solve otherwise-hard problems by training a neural network. Issues such as limited training size, model generalizability, etc are the key ones under investigation.

Modelling vehicle dynamics using Indian road traffic videos

Deep Neural Networks were employed to identify different vehicles in videos obtained from traffic junctions

Neural programming and program analysis

Automatically synthesising programs and analysing their behaviour is considered a holy grail of Artificial Intelligence. We work developing neural network based solutions towards automating these tasks. Our recent work has been published in venues such as FSE’16, AAAI’17 and AIED’18.

Network Consistent Data Association

Existing data association techniques mostly focus on matching pairs of data-point sets and then repeating this process along space-time to achieve long term correspondences. However, in many problems such as person re-identification, a set of data-points may be observed at multiple spatio-temporal locations and/or by multiple agents in a network and simply combining the local pairwise association results between sets of data-points often leads to inconsistencies over the global space-time horizons. In this paper, we propose a Novel Network Consistent Data Association (NCDA) framework formulated as an optimization problem that not only maintains consistency in association results across the network, but also improves the pairwise data association accuracies. The proposed NCDA can be solved as a binary integer program leading to a globally optimal solution and is capable of handling the challenging data-association scenario where the number of data-points varies across different sets of instances in the network. We also present an online implementation of NCDA method that can dynamically associate new observations to already observed data-points in an iterative fashion, while maintaining network consistency. We have tested both the batch and the online NCDA in two application areas—person re-identification and spatio-temporal cell tracking and observed consistent and highly accurate data association results in all the cases.

Fairness in an Algorithmic World

Siddharth’s recent work considers fairness from a computation lens. For example, one of his recent works shows that one can compute outcomes, in economic systems, which are efficient and fair at the same time, i.e., the seeming incompatible properties of efficiency and fairness can be achieved together. Given that fairness is a fundamental consideration in many resource-allocation problems, these results can potentially influence allocation policies in practical settings. Siddharth is also interested in developing fairness guarantees in machine-learning contexts such as clustering and classification.

Using Statistical Mechanics to understand depth in Deep Networks

Understanding the representational power of Restricted Boltzmann Machines (RBMs) with multiple layers is an ill-understood problem and is an area of active research. Motivated from the approach of Inherent Structure formalism (Stillinger & Weber, 1982), extensively used in analysing Spin Glasses, we propose a novel measure called Inherent Structure Capacity (ISC), which characterizes the representation capacity of a fixed architecture RBM by the expected number of modes of distributions emanating from the RBM with parameters drawn from a prior distribution. We introduce Lean RBMs, which are multi-layer RBMs where each layer can have at-most O(n) units with the number of visible units being n. We show that for every single layer RBM with Omega(n^{2+r}), r >= 0, hidden units there exists a two-layered lean RBM with Theta(n^2) parameters with the same ISC, establishing that 2 layer RBMs can achieve the same representational power as single-layer RBMs but using far fewer number of parameters. To the best of our knowledge, this is the first result which quantitatively establishes the need for layering.

Cross-Modal Retrieval

Due to increase in the number of sources of data, research in cross-modal matching is becoming an increasingly important area of research. It has several applications like match- ing text with image, matching near infra-red images with visible images (eg, for matching face images captured during night-time or low-light conditions to standard visible light images in the database), matching sketch images with pictures for forensic applications, etc. We are developing novel algorithms for this problem, which is extremely challenging, due to significant differences between data from different modalities. Specifically, we have developed a generalized semantic preserving hashing technique for cross-modal retrieval algorithms, which can work seamlessly for single and multi-label data, as well as in paired and unpaired scenarios. The algorithm obtained state-of-the-art performance in different applications.

Task agnostic Universal Adversarial Perturbations

Machine learning models are susceptible to adversarial perturbations: small changes to input that can cause large changes in output. It is also demonstrated that there exist input-agnostic perturbations, called universal adversarial perturbations, which can change the inference of target model on most of the data samples. However, existing methods to craft universal perturbations are (i) task specific, (ii) require samples from the training data distribution, and (iii) perform complex optimizations. Additionally, because of the data dependence, fooling ability of the crafted perturbations is proportional to the available training data. In this project, we present a novel, generalizable and data-free approach for crafting universal adversarial perturbations. Independent of the underlying task, our objective achieves fooling via corrupting the extracted features at multiple layers. Therefore, the proposed objective is generalizable to craft image-agnostic perturbations across multiple vision tasks such as object recognition, semantic segmentation, and depth estimation. In the practical setting of black-box attack scenario (when the attacker does not have access to the target model and it’s training data), we show that our objective outperforms the data dependent objectives to fool the learned models. Further, via exploiting simple priors related to the data distribution, our objective remarkably boosts the fooling ability of the crafted perturbations. Significant fooling rates achieved by our objective emphasize that the current deep learning models are now at an increased risk, since our objective generalizes across multiple tasks without the requirement of training data for crafting the perturbations.

Symmetry in Scalar Fields

Automatic detection of symmetry is a challenging problem because both the segmentation of the domain into potential symmetric segments and the correspondence between segments that are symmetric need to be determined. Moreover, real life data sets never exhibit perfect symmetry. Since the search space for locating symmetric segments is quite large, it is also important to design an algorithm that is computationally efficient. Domain experts are interested in studying important features that provide insights about the underlying scientific phenomena that is being analyzed.

We have developed four different methods to detect symmetry in scalar fields. We have also demonstrated applications to symmetry-aware transfer function design, symmetry-aware isosurface extraction, and symmetry-aware editing and rendering enhance traditional visualization methods. We believe that the methods we propose for symmetry detection will open new frontiers in analyzing structural similarity of scalar fields and more applications of symmetry detection will emerge.