Linyi Li

Assistant Professor, CS@SFU.
[firstnamelowercase]_[lastnamelowercase]@sfu.ca
[Curriculum Vitae]

I am Linyi LI, assistant professor in School of Computing Science, Simon Fraser University.

My research is in the intersection of machine learning, security, and software engineering. Specifically, I focus on: (1) building certifiably trustworthy deep learning systems, achieving certifiable robustness against noise perturbations [IJCAI 2019] [ICLR 2022a] [ICML 2022a] [SP 2023], certifiable robustness against semantic perturbations [CCS 2021] [ICML 2022b], certifiable robustness against poisoning attacks [ICLR 2022b], certifiable robustness against distributional shift [ICML 2022c], certifiable fairness [NeurIPS 2022], certifiable numerical reliability [ICSE 2023], etc. (2) Alignment and systematic evaluaton research in large language models. Previously, I did research in robustness of ensemble models [NeurIPS 2021], black-box attacks for deep learning [ICML 2021] [AISTATS 2021], and applications of machine learning in software testing [FSE 2020 Industry]. I am awarded Rising Stars in Data Science, AdvML Rising Star Award, and Wing Kai Cheng Fellowship; and I am the finalist of 2022 Qualcomm Innovation Fellowship and 2022 Two Sigma PhD Fellowship.

I obtained my PhD degree in Computer Science, University of Illinois Urbana-Champaign in 2023 advised by gorgeous Prof. Bo Li and awesome Prof. Tao Xie. I got my bachelor's degree from Department of Computer Science and Technology, Tsinghua University in 2018, where I did research on Web API Automated Testing, advised by Prof. Xiaoying Bai.

Interested in trustworthy machine learning / large language models?
Welcome to reach out! We have multiple PhD/Masters positions and collaboration opportunities available.
For undergrad/graduate students: Please send me an email with subject "[seek for (position/collaboration)]" to [first name in lowercase][email protected] or [first name in lowercase]_[last name in lowercase]@sfu.ca. English preferred. Response will be made when recruitment process starts and match positions available.
Those from underrepresented groups are particularly encouraged to apply!

News

  • [Nov, 2023] Introducing InfiCoder-Eval, a novel benchmark for open-world question-answering for code large language models.
  • [July, 2023] Our α,β-CROWN wins the neural network verification competition again!
  • [Dec, 2022] Our RANUM framework for assuring numerical reliability of deep neural networks is accepted by ICSE 2023.
  • [Oct, 2022] Happy to be selected as Rising Stars in Data Science at DSI, University of Chicago!
  • [Sept, 2022] I am co-organizing the workshop on Trustworthy and Socially Responsible Machine Learning at NeurIPS 2022. We invite submissions on any aspect of trustworthy and socially responsible machine learning.
  • [Sept, 2022] Five papers accepted to NeurIPS 2022. My co-first authored paper proposes a scalable method for certifying model's distributional fairness.
  • [Aug, 2022] Happy to receive 2022 AdvML Rising Star Award!
  • [Jun, 2022] We release a systematization of knowledge (SOK) paper (accepted by IEEE SP 2023) along with a toolkit for evaluating about 20 neural network verification approaches on GitHub.
  • [May, 2022] Three papers accepted by ICML 2022. We provide a tighter certification against L2 perturbations (link), a tighter certification for point cloud models (link), and an out-of-domain generalization certification (link). Look forward to seeing you in Baltimore in July 2022.
  • [May, 2022] Started internship at Microsoft Research New England on deep program synthesis - I am in Boston area this summer.
  • [Apr, 2022] Selected as finalist for 2022 Qualcomm Innovation Fellowship.
  • [Jan, 2022] Selected as finalist for 2022 Two Sigma PhD Fellowship.
  • [Jan, 2022] We propose practical robustness certification approaches for RL against evasion attacks (CROP, accepted by ICLR 2022) and poisoning attacks (COPA, accepted by ICLR 2022).
  • [Jan, 2022] Motivated by theoretical analysis, we propose DRT, a training approach for randomized smoothing that diversifies submodels within an ensemble to achieve state-of-the-art certified robustness. Check out our paper at ICLR 2022!

  • [Sept, 2021] Regularizing gradient similarity and model smoothness is sufficient to diversify sub-models in an ensemble, and thus leading to significant improvements on ensemble NN robustness. Details available in our paper at NeurIPS 2021.
  • [May, 2021] Simple downsampling combined with Progressive GAN can attack neural networks very efficiently. Details available in our paper at ICML 2021.
  • [May, 2021] We provide the first rigorous robustness certification on ImageNet against common image transformations including rotation and scaling! Paper will appear at CCS 2021.
  • [Jan, 2021] We will present a novel analysis of using non-linear projections for neural networks black-box attack at AISTATS 2021.
  • [Aug, 2020] Paper on clustering test steps leveraging NLP for automating software testing got accepted by ESEC/FSE'20 (Industry Track).
  • [Apr, 2020] Passed the Ph.D. Qualifying exam.
  • [Nov, 2019] Our team ranked 2nd in ICPC Mid-Central USA Regional Contest 2019.
  • [May, 2019] Paper on training provable robust NN via reference adversarial space got accepted by IJCAI'19.
  • [July, 2018] Graduated from Tsinghua University with Outstanding Underguaduate Award from the university and Excellence Undergraduate Award from the department.
  • [Feb, 2018] Recevied CS Ph.D admission offers from Carnegie Mellon University, University of Illinois at Urbana-Champaign and University of Wisconsin-Madison. Many thanks to everyone who helped my application!
  • [Sept, 2017] Finished summer internship at Carnegie Mellon University on neural network explaining, advised by Prof. Matt Fredrikson.
  • [Mar, 2017] Paper on Cloud API Testing got accepted by COMSPAC'17.
  • [Nov, 2015] Started to work with Prof. Xiaoying Bai on software testing.

Research

Currently, my research mainly aims at
  1. providing rigorous guarantees of various trustworthy properties (robustness, fairness, reliabiltiy, etc) for a given deep neural network system;
  2. improving such guaranteed trustworthiness for machine learning via strategic architecture design, dataset building, model training, post-processing, etc.
Click or touch to browse details.
(* denotes to equal contribution)

  • Linyi Li, Tao Xie, Bo Li
    SoK: Certified Robustness for Deep Neural Networks
    44th IEEE Symposium on Security and Privacy (SP 2023)
    [Full Version]   [Conference Version]   [Slides]   [Code]   [Leaderboard]  
    @inproceedings{li2023sok,
    author={Linyi Li and Tao Xie and Bo Li},
    title = {SoK: Certified Robustness for Deep Neural Networks},
    booktitle = {44th {IEEE} Symposium on Security and Privacy, {SP} 2023, San Francisco, CA, USA, 22-26 May 2023},
    publisher = {{IEEE}},
    year = {2023},
    }

    Topic: certified ML

    Summary A comprehensive systemization of knowledge on DNN certified robustness, including discussion on practical and theoretical implications, findings, main challenges, and future directions, accompanied with an open-source unified platform to evaluate 20+ representative approaches.

  • Linyi Li, Yuhao Zhang, Luyao Ren, Yingfei Xiong, Tao Xie
    Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects
    45th IEEE/ACM International Conference on Software Engineering (ICSE 2023)
    [Full Version]   [Conference Version]   [Slides]   [Code]  
    @inproceedings{li2023reliability,
    author={Linyi Li and Yuhao Zhang and Luyao Ren and Yingfei Xiong and Tao Xie},
    title = {Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects},
    booktitle = {45th International Conference on Software Engineering, {ICSE} 2023, Melbourne, Australia, 14-20 May 2023},
    publisher = {{IEEE/ACM}},
    year = {2023},
    }

    Topic: certified ML numerical reliability

    Summary An effective and efficient white-box framework for generic DNN architectures, named RANUM, for certifying numerical reliability (e.g., not output NaN or INF), generating failure-exhibiting system tests, and suggesting fixes, where RANUM is the first automated framework for the last two tasks.

  • Mintong Kang*, Linyi Li*, Maurice Weber, Yang Liu, Ce Zhang, Bo Li
    Certifying Some Distributional Fairness with Subpopulation Decomposition
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]   [Poster]  
    @inproceedings{kang2022certifying,
    title = {Certifying Some Distributional Fairness with Subpopulation Decomposition},
    author = {Mintong Kang and Linyi Li and Maurice Weber and Yang Liu and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: certified ML fairness

    Summary A practical and scalable certification approach to provide fairness bound for a given model when distribution shifts from training, based on subpopulation decomposition.

  • Linyi Li, Jiawei Zhang, Tao Xie, Bo Li
    Double Sampling Randomized Smoothing
    39th International Conference on Machine Learning (ICML 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    li2022double,
    title={Double Sampling Randomized Smoothing},
    author={Linyi Li and Jiawei Zhang and Tao Xie and Bo Li},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

    Summary A tighter certification approach for randomized smoothing, that for the first time circumvents the well-known curse of dimensionality under mild conditions by leveraging statistics from two strategically-chosen distributions.

  • Wenda Chu, Linyi Li, Bo Li
    TPC: Transformation-Specific Smoothing for Point Cloud Models
    39th International Conference on Machine Learning (ICML 2022)
    [Full Version]   [Code]  
    @inproceedings{
    chu2022tpc,
    title={TPC: Transformation-Specific Smoothing for Point Cloud Models},
    author={Wenda Chu and Linyi Li and Bo Li},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

    Summary By extending the methodology for certifying image classifiers against transformations, we provide state-of-the-art certification algorithms for point cloud models with detailed point cloud transformation analyses.

  • Maurice Weber, Linyi Li, Boxin Wang, Zhikuan Zhao, Bo Li, Ce Zhang
    Certifying Out-of-Domain Generalization for Blackbox Functions
    39th International Conference on Machine Learning (ICML 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    weber2022certifying,
    title={Certifying Out-of-Domain Generalization for Blackbox Functions},
    author={Maurice Weber and Linyi Li and Boxin Wang and Zhikuan Zhao and Bo Li and Ce Zhang},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

    Summary A scalable certification algorithm for model generalization against distributional shift which requires no assumption on the model's architecture, as long as the distributional shift is bounded by Hellinger distance, a type of f-divergence. Core methodology is based on the positive semidefinite property of Gramian matrix.

  • Fan Wu*, Linyi Li*, Chejian Xu, Huan Zhang, Bhavya Kailkhura, Krishnaram Kenthapadi, Ding Zhao, Bo Li
    COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Leaderboard]   [Code]  
    @inproceedings{
    wu2022copa,
    title={{COPA}: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks},
    author={Fan Wu and Linyi Li and Chejian Xu and Huan Zhang and Bhavya Kailkhura and Krishnaram Kenthapadi and Ding Zhao and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=psh0oeMSBiF}
    }

    Topic: certified ML deep reinforcement learning

    Summary The first approach for certifying deep RL robustness against offline training dataset perturbations, i.e., poisoning attacks, by aggregating over policies trained on partitioned datasets and policies for multiple time steps.

  • Zhuolin Yang*, Linyi Li*, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, Bo Li
    On the Certified Robustness for Ensemble Models and Beyond
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    yang2022on,
    title={On the Certified Robustness for Ensemble Models and Beyond},
    author={Zhuolin Yang and Linyi Li and Xiaojun Xu and Bhavya Kailkhura and Tao Xie and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=tUa4REjGjTf}
    }

    Topic: certified ML

    Summary Based on a curvature bound for randomized smoothing based classifiers, we prove that large confidence margin and gradient diversity are sufficient and necessary condition for certifiably robust ensembles. By regularizing these two factors, we acheive SOTA L2 certified robustness.

  • Fan Wu, Linyi Li, Zijian Huang, Yevgeniy Vorobeychik, Ding Zhao, Bo Li
    CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Leaderboard]   [Code]  
    @inproceedings{
    wu2022crop,
    title={{CROP}: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing},
    author={Fan Wu and Linyi Li and Zijian Huang and Yevgeniy Vorobeychik and Ding Zhao and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=HOjLHrlZhmx}
    }

    Topic: certified ML deep reinforcement learning

    Summary The first scalable approach for certifying deep RL robustness against state perturbations, by combining randomized smoothing with a set of trajectory-based search algorithms.

  • Zhuolin Yang*, Linyi Li*, Xiaojun Xu*, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin I. P. Rubinstein, Ce Zhang, Bo Li
    TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness
    Advances in Neural Information Processing Systems (NeurIPS) 2021
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{yangli2021trs,
    title = {TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness},
    author = {Zhuolin Yang and Linyi Li and Xiaojun Xu and Shiliang Zuo and Qian Chen and Pan Zhou and Benjamin I. P. Rubinstein and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 34 (NeurIPS 2021)},
    year = {2021}
    }

    Topic: robust ML

    Summary We prove the guaranteed correlation between model diversity and adversarial transferabiltiy given bounded model smoothness, which leads to a strong regularizer that achieves SOTA ensemble robustness against existing strong attacks.

  • Linyi Li*, Maurice Weber*, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, Bo Li
    TSS: Transformation-Specific Smoothing for Robustness Certification
    ACM Conference on Computer and Communications Security (CCS) 2021
    [Conference Version]   [Full Version]   [Code]   [Slides]  
    @inproceedings{li2021tss,
    title={TSS: Transformation-Specific Smoothing for Robustness Certification},
    author={Linyi Li and Maurice Weber and Xiaojun Xu and Luka Rimanic and Bhavya Kailkhura and Tao Xie and Ce Zhang and Bo Li},
    year={2021},
    booktitle={ACM Conference on Computer and Communications Security (CCS 2021)}
    }

    Topic: certified ML

    Summary Natural transformations such as rotation and scaling are common in the physical world. We propose the first scalable certification approach against natural transformations based on randomzied smoothing, rigorous Lipschitz analysis, and stratified sampling. For the first time, we certify non-trivial robustness (>30% certified robust accuracy) on the large-scale ImageNet dataset.

  • Linyi Li*, Zexuan Zhong*, Bo Li, Tao Xie
    Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space
    International Joint Conference on Artificial Intelligence (IJCAI) 2019
    [Paper]   [Code]  
    @inproceedings{li2019robustra,
    title = {Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space},
    author = {Li, Linyi and Zhong, Zexuan and Li, Bo and Xie, Tao},
    booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI 2019)},
    publisher = {International Joint Conferences on Artificial Intelligence Organization},
    pages = {4711--4717},
    year = {2019},
    month = {7},
    doi = {10.24963/ijcai.2019/654},
    url = {https://doi.org/10.24963/ijcai.2019/654}
    }

    Topic: certified ML

    Summary We propose a training method for achieving certified robustness by regularizing only within the reference adversarial space from a jointly trained model to alleviate the optimization hardness and achieve higher certified robustness.

(* denotes to equal contribution)

  1. Linyi Li, Tao Xie, Bo Li
    SoK: Certified Robustness for Deep Neural Networks
    44th IEEE Symposium on Security and Privacy (SP 2023)
    [Full Version]   [Conference Version]   [Slides]   [Code]   [Leaderboard]  
    @inproceedings{li2023sok,
    author={Linyi Li and Tao Xie and Bo Li},
    title = {SoK: Certified Robustness for Deep Neural Networks},
    booktitle = {44th {IEEE} Symposium on Security and Privacy, {SP} 2023, San Francisco, CA, USA, 22-26 May 2023},
    publisher = {{IEEE}},
    year = {2023},
    }

    Topic: certified ML

    Summary A comprehensive systemization of knowledge on DNN certified robustness, including discussion on practical and theoretical implications, findings, main challenges, and future directions, accompanied with an open-source unified platform to evaluate 20+ representative approaches.

  2. Linyi Li, Yuhao Zhang, Luyao Ren, Yingfei Xiong, Tao Xie
    Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects
    45th IEEE/ACM International Conference on Software Engineering (ICSE 2023)
    [Full Version]   [Conference Version]   [Slides]   [Code]  
    @inproceedings{li2023reliability,
    author={Linyi Li and Yuhao Zhang and Luyao Ren and Yingfei Xiong and Tao Xie},
    title = {Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects},
    booktitle = {45th International Conference on Software Engineering, {ICSE} 2023, Melbourne, Australia, 14-20 May 2023},
    publisher = {{IEEE/ACM}},
    year = {2023},
    }

    Topic: certified ML numerical reliability

    Summary An effective and efficient white-box framework for generic DNN architectures, named RANUM, for certifying numerical reliability (e.g., not output NaN or INF), generating failure-exhibiting system tests, and suggesting fixes, where RANUM is the first automated framework for the last two tasks.

  3. Mintong Kang*, Linyi Li*, Maurice Weber, Yang Liu, Ce Zhang, Bo Li
    Certifying Some Distributional Fairness with Subpopulation Decomposition
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]   [Poster]  
    @inproceedings{kang2022certifying,
    title = {Certifying Some Distributional Fairness with Subpopulation Decomposition},
    author = {Mintong Kang and Linyi Li and Maurice Weber and Yang Liu and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: certified ML fairness

    Summary A practical and scalable certification approach to provide fairness bound for a given model when distribution shifts from training, based on subpopulation decomposition.

  4. Linyi Li, Jiawei Zhang, Tao Xie, Bo Li
    Double Sampling Randomized Smoothing
    39th International Conference on Machine Learning (ICML 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    li2022double,
    title={Double Sampling Randomized Smoothing},
    author={Linyi Li and Jiawei Zhang and Tao Xie and Bo Li},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

    Summary A tighter certification approach for randomized smoothing, that for the first time circumvents the well-known curse of dimensionality under mild conditions by leveraging statistics from two strategically-chosen distributions.

  5. Fan Wu*, Linyi Li*, Chejian Xu, Huan Zhang, Bhavya Kailkhura, Krishnaram Kenthapadi, Ding Zhao, Bo Li
    COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Leaderboard]   [Code]  
    @inproceedings{
    wu2022copa,
    title={{COPA}: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks},
    author={Fan Wu and Linyi Li and Chejian Xu and Huan Zhang and Bhavya Kailkhura and Krishnaram Kenthapadi and Ding Zhao and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=psh0oeMSBiF}
    }

    Topic: certified ML deep reinforcement learning

    Summary The first approach for certifying deep RL robustness against offline training dataset perturbations, i.e., poisoning attacks, by aggregating over policies trained on partitioned datasets and policies for multiple time steps.

  6. Zhuolin Yang*, Linyi Li*, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, Bo Li
    On the Certified Robustness for Ensemble Models and Beyond
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    yang2022on,
    title={On the Certified Robustness for Ensemble Models and Beyond},
    author={Zhuolin Yang and Linyi Li and Xiaojun Xu and Bhavya Kailkhura and Tao Xie and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=tUa4REjGjTf}
    }

    Topic: certified ML

    Summary Based on a curvature bound for randomized smoothing based classifiers, we prove that large confidence margin and gradient diversity are sufficient and necessary condition for certifiably robust ensembles. By regularizing these two factors, we acheive SOTA L2 certified robustness.

  7. Zhuolin Yang*, Linyi Li*, Xiaojun Xu*, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin I. P. Rubinstein, Ce Zhang, Bo Li
    TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness
    Advances in Neural Information Processing Systems (NeurIPS) 2021
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{yangli2021trs,
    title = {TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness},
    author = {Zhuolin Yang and Linyi Li and Xiaojun Xu and Shiliang Zuo and Qian Chen and Pan Zhou and Benjamin I. P. Rubinstein and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 34 (NeurIPS 2021)},
    year = {2021}
    }

    Topic: robust ML

    Summary We prove the guaranteed correlation between model diversity and adversarial transferabiltiy given bounded model smoothness, which leads to a strong regularizer that achieves SOTA ensemble robustness against existing strong attacks.

  8. Jiawei Zhang*, Linyi Li*, Huichen Li, Xiaolu Zhang, Shuang Yang, Bo Li
    Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimation
    International Conference on Machine Learning (ICML) 2021
    [Conference Version]   [Full Version]   [Code]   [Slides]  
    @inproceedings{zhangli2021progressive,
    title = {Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimation},
    author = {Zhang, Jiawei and Li, Linyi and Li, Huichen and Zhang, Xiaolu and Yang, Shuang and Li, Bo},
    booktitle = {Proceedings of the 38th International Conference on Machine Learning (ICML 2021)},
    pages = {12479--12490},
    year = {2021},
    editor = {Meila, Marina and Zhang, Tong},
    volume = {139},
    series = {Proceedings of Machine Learning Research},
    month = {18--24 Jul},
    publisher = {PMLR},
    }

    Topic: attacks for ML

    Summary We systematically analyzed the gradient estimator that guides black-box attacks for DNNs, which reveals several key factors that can lead to more accurate gradient estimation with fewer queries. One way to realize these key factors is to conduct the attack with gradient estimation on a particularly scaled version of the image, which leads to the PSBA black-box attack with SOTA query effciency.

  9. Linyi Li*, Maurice Weber*, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, Bo Li
    TSS: Transformation-Specific Smoothing for Robustness Certification
    ACM Conference on Computer and Communications Security (CCS) 2021
    [Conference Version]   [Full Version]   [Code]   [Slides]  
    @inproceedings{li2021tss,
    title={TSS: Transformation-Specific Smoothing for Robustness Certification},
    author={Linyi Li and Maurice Weber and Xiaojun Xu and Luka Rimanic and Bhavya Kailkhura and Tao Xie and Ce Zhang and Bo Li},
    year={2021},
    booktitle={ACM Conference on Computer and Communications Security (CCS 2021)}
    }

    Topic: certified ML

    Summary Natural transformations such as rotation and scaling are common in the physical world. We propose the first scalable certification approach against natural transformations based on randomzied smoothing, rigorous Lipschitz analysis, and stratified sampling. For the first time, we certify non-trivial robustness (>30% certified robust accuracy) on the large-scale ImageNet dataset.

  10. Huichen Li*, Linyi Li*, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, Bo Li
    Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
    International Conference on Artificial Intelligence and Statistics (AISTATS) 2021
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{li2020nolinear,
    title={Nonlinear Gradient Estimation for Query Efficient Blackbox Attack},
    author={Huichen Li and Linyi Li and Xiaojun Xu and Xiaolu Zhang and Shuang Yang and Bo Li},
    year={2021},
    booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS 2021)},
    series = {Proceedings of Machine Learning Research},
    month = {13--15 Apr},
    publisher = {PMLR},
    }

    Topic: attacks for ML

    Summary We analyze the outcome of using nonlinear projections for black-box gradient-estimation-based attacks, which shows that proper nonlinear projections can help to improve the attack efficiency.

  11. Linyi Li, Zhenwen Li, Weijie Zhang, Jun Zhou, Pengcheng Wang, Jing Wu, Guanghua He, Xia Zeng, Yuetang Deng, Tao Xie
    Clustering Test Steps in Natural Language toward Automating Test Automation
    ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 2020, Industry Track
    [Paper]   [Video]  
    @inproceedings{li2020clustep,
    title = {Clustering Test Steps in Natural Language toward Automating Test Automation},
    author = {Li, Linyi and Li, Zhenwen and Zhang, Weijie and Zhou, Jun and Wang, Pengcheng and Wu, Jing and He, Guanghua and Zeng, Xia and Deng, Yuetang and Xie, Tao},
    booktitle = {Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering {(ESEC/FSE 2020)}},
    year = {2020},
    doi = {10.1145/3368089.3417067},
    url = {https://doi.org/10.1145/3368089.3417067}
    }

    Topic: ML for software testing

    Summary We provide an effective pipeline to cluster test steps in natural language and then synthesize executable test cases, deployed for WeChat testing.

  12. Linyi Li*, Zexuan Zhong*, Bo Li, Tao Xie
    Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space
    International Joint Conference on Artificial Intelligence (IJCAI) 2019
    [Paper]   [Code]  
    @inproceedings{li2019robustra,
    title = {Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space},
    author = {Li, Linyi and Zhong, Zexuan and Li, Bo and Xie, Tao},
    booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI 2019)},
    publisher = {International Joint Conferences on Artificial Intelligence Organization},
    pages = {4711--4717},
    year = {2019},
    month = {7},
    doi = {10.24963/ijcai.2019/654},
    url = {https://doi.org/10.24963/ijcai.2019/654}
    }

    Topic: certified ML

    Summary We propose a training method for achieving certified robustness by regularizing only within the reference adversarial space from a jointly trained model to alleviate the optimization hardness and achieve higher certified robustness.

(* denotes to equal contribution)

    2023

  1. Linyi Li
    Certifiably Trustworthy Deep Learning Systems at Scale
    Doctoral Thesis
    [Full Version]  
    @phdthesis{li2023thesis,
    title = {Certifiably Trustworthy Deep Learning Systems at Scale},
    author = {Linyi Li},
    year = 2023,
    month = {Oct},
    school = {University of Illinois Urbana-Champaign},
    type = {PhD thesis}
    }
  2. Zhangheng Li, Tianlong Chen, Linyi Li, Bo Li, Zhangyang Wang
    Can Pruning Improve Certified Robustness of Neural Networks?
    Transactions on Machine Learning Research (TMLR), 2023
    [Full Version]  
    @article{
    li2023can,
    title={Can Pruning Improve Certified Robustness of Neural Networks?},
    author={Zhangheng LI and Tianlong Chen and Linyi Li and Bo Li and Zhangyang Wang},
    journal={Transactions on Machine Learning Research},
    issn={2835-8856},
    year={2023},
    url={https://openreview.net/forum?id=6IFi2soduD},
    }

    Topic: certified ML pruning

  3. Linyi Li, Tao Xie, Bo Li
    SoK: Certified Robustness for Deep Neural Networks
    44th IEEE Symposium on Security and Privacy (SP 2023)
    [Full Version]   [Conference Version]   [Slides]   [Code]   [Leaderboard]  
    @inproceedings{li2023sok,
    author={Linyi Li and Tao Xie and Bo Li},
    title = {SoK: Certified Robustness for Deep Neural Networks},
    booktitle = {44th {IEEE} Symposium on Security and Privacy, {SP} 2023, San Francisco, CA, USA, 22-26 May 2023},
    publisher = {{IEEE}},
    year = {2023},
    }

    Topic: certified ML

  4. Linyi Li, Yuhao Zhang, Luyao Ren, Yingfei Xiong, Tao Xie
    Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects
    45th IEEE/ACM International Conference on Software Engineering (ICSE 2023)
    [Full Version]   [Conference Version]   [Slides]   [Code]  
    @inproceedings{li2023reliability,
    author={Linyi Li and Yuhao Zhang and Luyao Ren and Yingfei Xiong and Tao Xie},
    title = {Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects},
    booktitle = {45th International Conference on Software Engineering, {ICSE} 2023, Melbourne, Australia, 14-20 May 2023},
    publisher = {{IEEE/ACM}},
    year = {2023},
    }

    Topic: certified ML numerical reliability

  5. Jiawei Zhang, Linyi Li, Ce Zhang, Bo Li
    CARE: Certifiably Robust Learning with Reasoning via Variational Inference
    First IEEE Conference on Secure and Trustworthy Machine Learning (SatML 2023)
    [Full Version]   [Conference Version]  
    @inproceedings{
    zhang2023care,
    title={{CARE}: Certifiably Robust Learning with Reasoning via Variational Inference},
    author={Jiawei Zhang and Linyi Li and Ce Zhang and Bo Li},
    booktitle={First IEEE Conference on Secure and Trustworthy Machine Learning},
    year={2023},
    url={https://openreview.net/forum?id=1n6oWTTV1n}
    }

    Topic: certified ML reasoning

  6. Mintong Kang, Linyi Li, Bo Li
    FaShapley: Fast and Approximated Shapley Based Model Pruning Towards Certifiably Robust DNNs
    First IEEE Conference on Secure and Trustworthy Machine Learning (SatML 2023)
    [Conference Version]  
    @inproceedings{
    kang2023fashapley,
    title={FaShapley: Fast and Approximated Shapley Based Model Pruning Towards Certifiably Robust {DNN}s},
    author={Mintong Kang and Linyi Li and Bo Li},
    booktitle={First IEEE Conference on Secure and Trustworthy Machine Learning},
    year={2023},
    url={https://openreview.net/forum?id=mJF9_Fs52ut}
    }

    Topic: certified ML pruning

  7. 2022

  8. Mintong Kang*, Linyi Li*, Maurice Weber, Yang Liu, Ce Zhang, Bo Li
    Certifying Some Distributional Fairness with Subpopulation Decomposition
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]   [Poster]  
    @inproceedings{kang2022certifying,
    title = {Certifying Some Distributional Fairness with Subpopulation Decomposition},
    author = {Mintong Kang and Linyi Li and Maurice Weber and Yang Liu and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: certified ML fairness

  9. Xiaojun Xu, Linyi Li, Bo Li
    LOT: Layer-wise Orthogonal Training on Improving \(\ell_2\) Certified Robustness
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]  
    @inproceedings{xu2022lot,
    title = {LOT: Layer-wise Orthogonal Training on Improving l2 Certified Robustness},
    author = {Xiaojun Xu and Linyi Li and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: certified ML

  10. Bhaskar Ray Chaudhury, Linyi Li, Mintong Kang, Bo Li, Ruta Mehta
    Fairness in Federated Learning via Core-Stability
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]   [Poster]  
    @inproceedings{bhaskar2022fairness,
    title = {Fairness in Federated Learning via Core-Stability},
    author = {Bhaskar Ray Chaudhury and Linyi Li and Mintong Kang and Bo Li and Ruta Mehta},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: fairness

  11. Huan Zhang*, Shiqi Wang*, Kaidi Xu*, Linyi Li, Bo Li, Suman Jana, Cho-Jui Hsieh, J. Zico Kolter
    General Cutting Planes for Bound-Propagation-Based Neural Network Verification
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]   [Poster]  
    @inproceedings{zhang2022general,
    title = {General Cutting Planes for Bound-Propagation-Based Neural Network Verification},
    author = {Huan Zhang and Shiqi Wang and Kaidi Xu and Linyi Li and Bo Li and Suman Jana and Cho-Jui Hsieh and J. Zico Kolter},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: certified ML

  12. Zhuolin Yang*, Zhikuan Zhao*, Boxin Wang, Jiawei Zhang, Linyi Li, Hengzhi Pei, Bojan Karlaš, Ji Liu, Heng Guo, Ce Zhang, Bo Li
    Improving Certified Robustness via Statistical Learning with Logical Reasoning
    Advances in Neural Information Processing Systems (NeurIPS) 2022
    [Full Version]   [Conference Version]   [Code]  
    @inproceedings{yang2022improving,
    title = {Improving Certified Robustness via Statistical Learning with Logical Reasoning},
    author = {Zhuolin Yang and Zhikuan Zhao and Boxin Wang and Jiawei Zhang and Linyi Li and Hengzhi Pei and Bojan Karlaš and Ji Liu and Heng Guo and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
    year = {2022}
    }

    Topic: certified ML reasoning

  13. Hanjiang Hu, Zuxin Liu, Linyi Li, Jiacheng Zhu, Ding Zhao
    Robustness Certification of Visual Perception Models via Camera Motion Smoothing
    6th Annual Conference on Robot Learning (CoRL 2022)
    [Paper]   [Forum]   [Code]  
    @inproceedings{
    hu2022robustness,
    title={Robustness Certification of Visual Perception Models via Camera Motion Smoothing},
    author={Hanjiang Hu and Zuxin Liu and Linyi Li and Jiacheng Zhu and Ding Zhao},
    booktitle={6th Annual Conference on Robot Learning},
    year={2022},
    url={https://openreview.net/forum?id=uUxDTZK3o3X}
    }

    Topic: certified ML

  14. Linyi Li, Jiawei Zhang, Tao Xie, Bo Li
    Double Sampling Randomized Smoothing
    39th International Conference on Machine Learning (ICML 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    li2022double,
    title={Double Sampling Randomized Smoothing},
    author={Linyi Li and Jiawei Zhang and Tao Xie and Bo Li},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

  15. Wenda Chu, Linyi Li, Bo Li
    TPC: Transformation-Specific Smoothing for Point Cloud Models
    39th International Conference on Machine Learning (ICML 2022)
    [Full Version]   [Code]  
    @inproceedings{
    chu2022tpc,
    title={TPC: Transformation-Specific Smoothing for Point Cloud Models},
    author={Wenda Chu and Linyi Li and Bo Li},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

  16. Maurice Weber, Linyi Li, Boxin Wang, Zhikuan Zhao, Bo Li, Ce Zhang
    Certifying Out-of-Domain Generalization for Blackbox Functions
    39th International Conference on Machine Learning (ICML 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    weber2022certifying,
    title={Certifying Out-of-Domain Generalization for Blackbox Functions},
    author={Maurice Weber and Linyi Li and Boxin Wang and Zhikuan Zhao and Bo Li and Ce Zhang},
    booktitle={39th International Conference on Machine Learning (ICML 2022)},
    year={2022},
    }

    Topic: certified ML

  17. Fan Wu*, Linyi Li*, Chejian Xu, Huan Zhang, Bhavya Kailkhura, Krishnaram Kenthapadi, Ding Zhao, Bo Li
    COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Leaderboard]   [Code]  
    @inproceedings{
    wu2022copa,
    title={{COPA}: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks},
    author={Fan Wu and Linyi Li and Chejian Xu and Huan Zhang and Bhavya Kailkhura and Krishnaram Kenthapadi and Ding Zhao and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=psh0oeMSBiF}
    }

    Topic: certified ML deep reinforcement learning

  18. Zhuolin Yang*, Linyi Li*, Xiaojun Xu, Bhavya Kailkhura, Tao Xie, Bo Li
    On the Certified Robustness for Ensemble Models and Beyond
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{
    yang2022on,
    title={On the Certified Robustness for Ensemble Models and Beyond},
    author={Zhuolin Yang and Linyi Li and Xiaojun Xu and Bhavya Kailkhura and Tao Xie and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=tUa4REjGjTf}
    }

    Topic: certified ML

  19. Fan Wu, Linyi Li, Zijian Huang, Yevgeniy Vorobeychik, Ding Zhao, Bo Li
    CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
    10th International Conference on Learning Representations (ICLR 2022)
    [Conference Version]   [Full Version]   [Leaderboard]   [Code]  
    @inproceedings{
    wu2022crop,
    title={{CROP}: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing},
    author={Fan Wu and Linyi Li and Zijian Huang and Yevgeniy Vorobeychik and Ding Zhao and Bo Li},
    booktitle={International Conference on Learning Representations},
    year={2022},
    url={https://openreview.net/forum?id=HOjLHrlZhmx}
    }

    Topic: certified ML deep reinforcement learning

  20. Ripon Saha, Akira Ura, Sonal Mahajan, Chenguang Zhu, Linyi Li, Yang Hu, Hiroaki Yoshida, Sarfraz Khurshid, Mukul R. Prasad
    SapientML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions
    44th International Conference on Software Engineering (ICSE 2022)
    [Conference Version]   [Full Version]  
    @inproceedings{saha2022sapientml,
    title={SapientML: synthesizing machine learning pipelines by learning from human-written solutions},
    author={Ripon Saha, Akira Ura, Sonal Mahajan, Chenguang Zhu, Linyi Li, Yang Hu, Hiroaki Yoshida, Sarfraz Khurshid, Mukul R. Prasad},
    booktitle={2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE)},
    year={2022},
    organization={IEEE}
    }

    Topic: autoML

  21. 2021

  22. Zhuolin Yang*, Linyi Li*, Xiaojun Xu*, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin I. P. Rubinstein, Ce Zhang, Bo Li
    TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness
    Advances in Neural Information Processing Systems (NeurIPS) 2021
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{yangli2021trs,
    title = {TRS: Transferability Reduced Ensemble via Promoting Gradient Diversity and Model Smoothness},
    author = {Zhuolin Yang and Linyi Li and Xiaojun Xu and Shiliang Zuo and Qian Chen and Pan Zhou and Benjamin I. P. Rubinstein and Ce Zhang and Bo Li},
    booktitle = {Advances in Neural Information Processing Systems 34 (NeurIPS 2021)},
    year = {2021}
    }

    Topic: robust ML

  23. Jiawei Zhang*, Linyi Li*, Huichen Li, Xiaolu Zhang, Shuang Yang, Bo Li
    Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimation
    International Conference on Machine Learning (ICML) 2021
    [Conference Version]   [Full Version]   [Code]   [Slides]  
    @inproceedings{zhangli2021progressive,
    title = {Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimation},
    author = {Zhang, Jiawei and Li, Linyi and Li, Huichen and Zhang, Xiaolu and Yang, Shuang and Li, Bo},
    booktitle = {Proceedings of the 38th International Conference on Machine Learning (ICML 2021)},
    pages = {12479--12490},
    year = {2021},
    editor = {Meila, Marina and Zhang, Tong},
    volume = {139},
    series = {Proceedings of Machine Learning Research},
    month = {18--24 Jul},
    publisher = {PMLR},
    }

    Topic: attacks for ML

  24. Linyi Li*, Maurice Weber*, Xiaojun Xu, Luka Rimanic, Bhavya Kailkhura, Tao Xie, Ce Zhang, Bo Li
    TSS: Transformation-Specific Smoothing for Robustness Certification
    ACM Conference on Computer and Communications Security (CCS) 2021
    [Conference Version]   [Full Version]   [Code]   [Slides]  
    @inproceedings{li2021tss,
    title={TSS: Transformation-Specific Smoothing for Robustness Certification},
    author={Linyi Li and Maurice Weber and Xiaojun Xu and Luka Rimanic and Bhavya Kailkhura and Tao Xie and Ce Zhang and Bo Li},
    year={2021},
    booktitle={ACM Conference on Computer and Communications Security (CCS 2021)}
    }

    Topic: certified ML

  25. Huichen Li*, Linyi Li*, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, Bo Li
    Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
    International Conference on Artificial Intelligence and Statistics (AISTATS) 2021
    [Conference Version]   [Full Version]   [Code]  
    @inproceedings{li2020nolinear,
    title={Nonlinear Gradient Estimation for Query Efficient Blackbox Attack},
    author={Huichen Li and Linyi Li and Xiaojun Xu and Xiaolu Zhang and Shuang Yang and Bo Li},
    year={2021},
    booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS 2021)},
    series = {Proceedings of Machine Learning Research},
    month = {13--15 Apr},
    publisher = {PMLR},
    }

    Topic: attacks for ML

  26. 2020

  27. Linyi Li, Zhenwen Li, Weijie Zhang, Jun Zhou, Pengcheng Wang, Jing Wu, Guanghua He, Xia Zeng, Yuetang Deng, Tao Xie
    Clustering Test Steps in Natural Language toward Automating Test Automation
    ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 2020, Industry Track
    [Paper]   [Video]  
    @inproceedings{li2020clustep,
    title = {Clustering Test Steps in Natural Language toward Automating Test Automation},
    author = {Li, Linyi and Li, Zhenwen and Zhang, Weijie and Zhou, Jun and Wang, Pengcheng and Wu, Jing and He, Guanghua and Zeng, Xia and Deng, Yuetang and Xie, Tao},
    booktitle = {Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering {(ESEC/FSE 2020)}},
    year = {2020},
    doi = {10.1145/3368089.3417067},
    url = {https://doi.org/10.1145/3368089.3417067}
    }

    Topic: ML for software testing

  28. 2019

  29. Linyi Li*, Zexuan Zhong*, Bo Li, Tao Xie
    Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space
    International Joint Conference on Artificial Intelligence (IJCAI) 2019
    [Paper]   [Code]  
    @inproceedings{li2019robustra,
    title = {Robustra: Training Provable Robust Neural Networks over Reference Adversarial Space},
    author = {Li, Linyi and Zhong, Zexuan and Li, Bo and Xie, Tao},
    booktitle = {Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI 2019)},
    publisher = {International Joint Conferences on Artificial Intelligence Organization},
    pages = {4711--4717},
    year = {2019},
    month = {7},
    doi = {10.24963/ijcai.2019/654},
    url = {https://doi.org/10.24963/ijcai.2019/654}
    }

    Topic: certified ML

  30. 2018

  31. Klas Leino, Shayak Sen, Anupam Datta, Matt Fredrikson, Linyi Li
    Influence-Directed Explanations for Deep Convolutional Networks
    IEEE International Test Conference (ITC) 2018
    [Paper]  
    @inproceedings{leino2018influence,
    author={Leino, Klas and Sen, Shayak and Datta, Anupam and Fredrikson, Matt and Li, Linyi},
    booktitle={2018 IEEE International Test Conference (ITC)},
    title={Influence-Directed Explanations for Deep Convolutional Networks},
    year={2018},
    pages={1-8},
    }

    Topic: intepretable ML undergrad research

  32. 2017

  33. Junyi Wang, Xiaoying Bai, Linyi Li, Zhicheng Ji, Haoran Ma
    A Model-Based Framework For Cloud API Testing
    IEEE 41st Annual Computer Software and Applications Conference (COMPSAC) 2017
    [Paper]  
    @inproceedings{wang2017model,
    author={Wang, Junyi and Bai, Xiaoying and Li, Linyi and Ji, Zhicheng and Ma, Haoran},
    booktitle={2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC)},
    title={A Model-Based Framework for Cloud API Testing},
    year={2017},
    volume={2},
    pages={60-65},
    doi={10.1109/COMPSAC.2017.24},
    ISSN={0730-3157},
    month={July},
    }

    Topic: software testing undergrad research

  34. Junyi Wang, Xiaoying Bai, Haoran Ma, Linyi Li, Zhicheng Ji
    Cloud API Testing
    IEEE International Conference on Software Verification and Validation Workshops (ICSTW) 2017
    [Paper]  
    @inproceedings{wang2017cloud,
    title={Cloud API testing},
    author={Wang, Junyi and Bai, Xiaoying and Ma, Haoran and Li, Linyi and Ji, Zhicheng},
    booktitle={2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)},
    pages={385--386},
    year={2017},
    organization={IEEE}
    }

    Topic: software testing undergrad research

Preprints can be found in Google Scholar profile.

Miscellaneous

  • I love traveling, geography, and languages especially Chinese Phonology. I admire Yuen Ren Chao.
  • Sometimes I play programming contests for fun.
  • I love VERY VERY spicy 🌶 food :)
  • I was born and spent childhood in Zhangjiajie, China. I lived in Changsha, China before college.
  • I am a Northern Tujia. In Tujia Language: Ngaf Bifzivkar.
Last Updated: Aug 14, 2023