Publications

2023

1. NeurIPS 2023 Spotlight

   Learning from Active Human Involvement through Proxy Value Propagation

   Zhenghao Peng, Wenjie Mo, Chenda Duan, and 2 more authors

   2023

   Abs Bib

   Summary Learning from active human involvement enables the human subject to actively intervene and demonstrate to the AI agent during training. It brings safety and AI alignment to the learning process. In this work, we propose a new reward-free active human involvement method called Proxy Value Propagation for policy optimization. Our key insight is that a proxy value function can be designed to express human intents, wherein state-action pairs in the human demonstration are labeled with high values, while those agents’ actions that are intervened receive low values. Through the TD-learning framework, labeled values of demonstrated state-action pairs are further propagated to other unlabeled data generated from agents’ exploration. The proxy value function thus induces a policy that faithfully emulates human behaviors. Human-in-the-loop experiments show the generality and efficiency of our method. With minimum modification to existing reinforcement learning algorithms, our method can learn to solve continuous and discrete control tasks with various human control devices, including the challenging task of driving in Grand Theft Auto V. Demo video and code are in the supplementary material and will be made public.

   @book{chenda_rl1,
     teaser_video = {cover_HACO.mp4},
     title = {Learning from Active Human Involvement through Proxy Value Propagation},
     author = {Peng, Zhenghao and Mo, Wenjie and Duan, Chenda and Li, Quanyi and Zhou, Bolei},
     year = {2023}
   }

2. NeurIPS 2023

   ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling

   Quanyi Li, Zhenghao Peng, Lan Feng, and 3 more authors

   2023

   Abs Bib HTML

   Summary Large-scale driving datasets such as Waymo Open Dataset and nuScenes substantially accelerate autonomous driving research, especially for perception tasks such as 3D detection and trajectory forecasting. Since the driving logs in these datasets contain HD maps and detailed object annotations which accurately reflect the real-world complexity of traffic behaviors, we can harvest a massive number of complex traffic scenarios and recreate their digital twins in simulation. Compared to the hand-crafted scenarios often used in existing simulators, data-driven scenarios collected from the real world can facilitate many research opportunities in machine learning and autonomous driving. In this work, we present ScenarioNet, an open-source platform for large-scale traffic scenario modeling and simulation. ScenarioNet defines a unified scenario description format and collects a large-scale repository of real-world traffic scenarios from the heterogeneous data in various driving datasets including Waymo, nuScenes, Lyft L5, and nuPlan datasets. These scenarios can be further replayed and interacted with in multiple views from Bird-Eye-View layout to realistic 3D rendering in MetaDrive simulator. This provides a benchmark for evaluating the safety of autonomous driving stacks in simulation before their real-world deployment. We further demonstrate the strengths of ScenarioNet on large-scale scenario generation, imitation learning, and reinforcement learning in both single-agent and multi-agent settings.

   @book{chenda_rl2,
     teaser = {cover_ScenarioNet.gif},
     title = {ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling},
     author = {Li, Quanyi and Peng, Zhenghao and Feng, Lan and Liu, Zhizheng and Mo, Chenda Duanand Wenjie and Zhou, Bolei},
     year = {2023}
   }

2022

1. European Neuropsychopharmacology

   Cell type deconvolution of bulk blood RNA-Seq to reveal biological insights of neuropsychiatric disorders

   Toni Boltz, Tommer Schwarz, Merel Bot, and 9 more authors

   European Neuropsychopharmacology 2022

   Abs Bib HTML

   Summary Genome-wide association studies (GWAS) have uncovered susceptibility loci associated with psychiatric disorders like bipolar disorder (BP) and schizophrenia (SCZ). However, most of these loci are in non-coding regions of the genome with unknown causal mechanisms of the link between genetic variation and disease risk. Expression quantitative trait loci (eQTL) analysis of bulk tissue is a common approach to decipher underlying mechanisms, though this can obscure cell-type specific signals thus masking trait-relevant mechanisms. While single-cell sequencing can be prohibitively expensive in large cohorts, computationally inferred cell type proportions and cell type gene expression estimates have the potential to overcome these problems and advance mechanistic studies. Using bulk RNA-Seq from 1,730 samples derived from whole blood in a cohort ascertained for individuals with BP and SCZ this study estimated cell type proportions and their relation with disease status and medication. We found between 2,875 and 4,629 eGenes for each cell type, including 1,211 eGenes that are not found using bulk expression alone. We performed a colocalization test between cell type eQTLs and various traits and identified hundreds of associations between cell type eQTLs and GWAS loci that are not detected in bulk eQTLs. Finally, we investigated the effects of lithium use on cell type expression regulation and found examples of genes that are differentially regulated dependent on lithium use. Our study suggests that computational methods can be applied to large bulk RNA-Seq datasets of non-brain tissue to identify disease-relevant, cell type specific biology of psychiatric disorders and psychiatric medication.

   @article{chenda_bioinfo2,
     teaser = {cover_celltype.png},
     title = {Cell type deconvolution of bulk blood RNA-Seq to reveal biological insights of neuropsychiatric disorders},
     journal = {European Neuropsychopharmacology},
     year = {2022},
     author = {Boltz, Toni and Schwarz, Tommer and Bot, Merel and Hou, Kangcheng and Caggiano, Christa and Lapinska, Sandra and Duan, Chenda and Boks, Marco P. and Kahn, Rene S. and Zaitlen, Noah and Pasaniuc, Bogdan and Ophoff, Roel},
     keywords = {RNA-seq, eQTL mapping, association testing, low coverage}
   }

2. HGG Advance

   Powerful eQTL mapping through low-coverage RNA sequencing

   Tommer Schwarz, Toni Boltz, Kangcheng Hou, and 7 more authors

   Human Genetics and Genomics Advances 2022

   Abs Bib HTML

   Summary Mapping genetic variants that regulate gene expression (eQTL mapping) in large-scale RNA sequencing (RNA-seq) studies is often employed to understand functional consequences of regulatory variants. However, the high cost of RNA-seq limits sample size, sequencing depth, and, therefore, discovery power in eQTL studies. In this work, we demonstrate that, given a fixed budget, eQTL discovery power can be increased by lowering the sequencing depth per sample and increasing the number of individuals sequenced in the assay. We perform RNA-seq of whole-blood tissue across 1,490 individuals at low coverage (5.9 million reads/sample) and show that the effective power is higher than that of an RNA-seq study of 570 individuals at moderate coverage (13.9 million reads/sample). Next, we leverage synthetic datasets derived from real RNA-seq data (50 million reads/sample) to explore the interplay of coverage and number individuals in eQTL studies, and show that a 10-fold reduction in coverage leads to only a 2.5-fold reduction in statistical power to identify eQTLs. Our work suggests that lowering coverage while increasing the number of individuals in RNA-seq is an effective approach to increase discovery power in eQTL studies.

   @article{chenda_bioinfo1,
     teaser = {cover_eqtl.jpg},
     title = {Powerful eQTL mapping through low-coverage RNA sequencing},
     journal = {Human Genetics and Genomics Advances},
     volume = {3},
     number = {3},
     pages = {100103},
     year = {2022},
     issn = {2666-2477},
     doi = {https://doi.org/10.1016/j.xhgg.2022.100103},
     author = {Schwarz, Tommer and Boltz, Toni and Hou, Kangcheng and Bot, Merel and Duan, Chenda and Loohuis, Loes Olde and Boks, Marco P. and Kahn, René S. and Ophoff, Roel A. and Pasaniuc, Bogdan},
     keywords = {RNA-seq, eQTL mapping, association testing, low coverage}
   }