Kyriakos G. Vamvoudakis - Research Projects
Our research draws from the areas of control theory, cyber-physical systems, game theory, and reinforcement learning. Our recent research interests lie in the design of robust, safe, and secure multi-agent networked systems. Below is a brief description of some recent research projects.
Towards Deceptive and Domain-Specific Cyber-Physical Honeypots
Funding Source: NSF SATC-2231651
Collaborator: S. Zonouz (GaTech)
Cyber-physical critical infrastructures provide management and control capabilities for mission-critical utilities such as power grids. Programmable logic controllers (PLCs) play a key role as they serve as a convenient bridge between the cyber and the physical worlds. PLCs' critical roles have made them the target of sophisticated cyberattacks that are designed to disrupt their operation, which creates both social unrest and financial losses. In this context, honeypots have been shown to be highly valuable tools for collecting data to better understand the many different strategies and objectives of the attackers. The project's novelty is to develop a new domain-specific stealthy honeypot for cyber-physical critical infrastructures and specifically PLC controllers. The solutions allow for active data collection using autonomous interactions with the malware to activate its malicious capabilities The project's broader significance and importance are to provide guidelines for researchers and practitioners looking to incorporate honeypots and security methods into cyber-physical systems (CPS) and embedded controllers.
For complete stealth, the solutions of this project: (i) leverage air-gapped observations of the malware behavior through physical side channels such as PLC processor power signal produced by the on-device malicious code execution and (ii) use mathematical models and physics-informed neural networks for a realistic emulation of the physical dynamics and to provide a misleading physical process interface to the PLC input-output ports. The research outcomes will address the above-mentioned semantic gap via an automated binary reverse engineering of the malicious controller code to extract the high-level adversarial objectives from low-level controller software execution traces. This enables the classification of adversaries dynamically using online data-driven meta-learning algorithms.
For more details see our published work.
An Integrated Reactive and Proactive Adversarial Learning for Cyber-Physical-Human Systems
Funding Source: NSF CPS-2227185, NSF CPS-2227153
Collaborator: Z. P. Jiang (NYU)
The gradual deployment of self-driving cars will inevitably lead to the emergence of a new important class of cyber-physical-human systems where autonomous vehicles interact with human-driven vehicles via on-board sensors or vehicle-to-vehicle communications. Reinforcement learning along with control theory can help meet the safety requirements for real-time decision making and Level 5 autonomy in self-driving vehicles. However, it is widely known that conventional reinforcement learning policies are vulnerable to adversarial or non-adversarial perturbations to their observations, similar to adversarial examples for classifiers and/or reward (packet) drops of the learning. Such issues are exacerbated by concerns of addressing resiliency as the use of open communication and control platforms for autonomy becomes essential, and as the industry continues to invest in such systems. Decision making mechanisms, designed to incorporate agility with the help of reinforcement learning, allow self-adaptation, self-healing, and self-optimization. This research will contribute and unify the body of knowledge of several diverse fields including reinforcement learning, security, automatic control, and transportation for resilient autonomy with humans-in-the-loop.
This project will leverage proactive switching policies that aim (i) to provide robustness to adversarial inputs and reward drops in the closed-loop reinforcement learning mechanisms, (ii) to increase the cost of manipulation by deception, (iii) to limit the exposure of vulnerable actions and observations, and (iv) to provide stability, optimality, and robustness guarantees. It will contribute to future engineering curricula, pursue a substantial integration of research and education, and provide opportunities to engage students from underrepresented groups.
For more details see our published work.
Learning-based Motion Planning Framework for Dynamic and Unknown Environments
Funding Source: US ARMY CO
Substantial improvements in artificial intelligence, computing resources, and software tools have
enabled tremendous capabilities to mobile robots and autonomous systems. The problem of motion
planning and navigation is a core topic in robotics and autonomous vehicles, as the majority of
robotic applications require safe path planning and obstacle avoidance. Ideally, a solution to this
problem considers collision-free navigation in dynamic environments, computationally affordable
algorithms for real-time implementation, and optimal control strategies. Safe motion planning
of mobile robots in dynamic environments is a challenging task with applications in autonomous
driving and robotic navigation in pedestrian environments.
This project will develop a safe, model-free, kinodynamic motion planning framework that can be
implemented online. It will consist of five phases: (i) the theoretical and algorithmic
formulation; (ii) the rigorous Lyapunov-based stability proof; (iii) the optimality/robustness quantification; (iv) the safety-obstacle awareness; and (v) the implementation. For more details see our published work.
Wildland Fire Observation, Management, and Evacuation using Intelligent Collaborative Flying and Ground Systems
Funding Source: NSF CPS-2038589, NSF CPS-2039026, NSF CPS-2038759, NSF CPS-2038741
Collaborators: F. Afghah and several others from the Northern Arizona University, A. Watts (Nevada System of Higher Education, Desert Research Institute), and J. Coen (University Corporation for Atmospheric Research)
Increasing wildfire costs---a reflection of climate variability and development within wildlands---drive calls for new national capabilities to manage wildfires. The great potential of unmanned aerial systems (UAS) has not yet been fully utilized in this domain due to the lack of holistic, resilient, flexible, and cost-effective monitoring protocols. This project will develop UAS-based fire management strategies to use autonomous unmanned aerial vehicles (UAVs) in an optimal, efficient, and safe way to assist the first responders during the fire detection, management, and evacuation stages. This proposal's objective is to develop an integrated framework satisfying unmet wildland fire management needs, with key advances in scientific and engineering methods by using a network of low-cost and small autonomous UAVs along with ground vehicles during different stages of fire management operations including: (i) early detection in remote and forest areas using autonomous UAVs; (ii) fast active geo-mapping of the fire heat map on flying drones; (iii) real-time video streaming of the fire spread; and (iv) finding optimal evacuation paths using autonomous UAVs to guide the ground vehicles and firefighters for fast and safe evacuation.
This project will advance the frontier of disaster management by developing: (i) an innovative drone-based forest fire detection and monitoring technology for rapid intervention in hard-to-access areas with minimal human intervention to protect firefighter lives; (ii) multi-level fire modeling to offer strategic, event-scale, and new on-board, low-computation tactics using fast fire mapping from UAVs; and (iii) a bounded reasoning-based planning mechanism where the UAVs identify the fastest and safest evacuation roads for firefighters and fire-trucks in highly dynamic and uncertain dangerous zones. The developed technologies will be translational to a broad range of applications such as disaster (flooding, fire, mud slides, terrorism) management, where quick search, surveillance, and responses are required with limited human interventions. This project will also contribute to future engineering curricula and pursue a substantial integration of research and education while also engaging female and underrepresented minority students, developing hands-on research experiments for K-12 students.
For more details see our published work.
Safety Fault Taxonomy and Fault Detection for CPS
Funding Source: CMU/SEI
Due to the autonomous nature of learning-enabled CPS, it has become increasingly difficult and expensive to design evaluation and testing scenarios
that verify the robustness of their operation and the satisfiability of the various specifications.
This, in turn, hinders the development of algorithms that allow model-free learning strategies to be implemented while the system remains within given safe areas. Thus, the problem of assured (or safe) autonomy has been in the forefront of research activities in both the control and the learning communities.
It has become apparent that alongside the systems themselves, evaluation strategies need to be informed by learning components as well, allowing for black-box paradigms to be introduced as solutions to the verification problem.
The objective of this project is to develop mechanisms for increased safety, security and fault tolerance of cyber-physical systems with increased autonomy capabilities. We will investigate potential faults that can occur due to the existence of learning mechanisms in the system due to the structure of those mechanisms or the opportunities of attack they provide to malicious agents. We will further analyze those faults in terms of the effects they have on the system, in order to design a complete taxonomy and hierarchy that will be integrated in the SAE AADL (Architecture Analysis and Design Language). Subsequently, we propose various detection mechanisms for faults in the learning-based components, as well as isolation rules that will be able to remove the potentially corrupted ones and maintain safe operation of the system. Finally, we will validate the proposed approaches in a unmanned aerial vehicle experimental platform. Our goal is to integrate continuous-time system-theoretic methods of learning and security with logic-based contracts that will express the safety, security and robustness specifications for the platform in a formal way.
For more details see our published work.
Secure and Safe Assured Autonomy
Funding Source: NASA ULI
Collaborators: A. Homaifar, and several others from NCAT, Purdue, Aurora Flight Sciences, Alakai Technologies Corporation, General Atomics Aeronautical Systems, Northrop Grumman Corporation
The last few decades have witnessed uninterrupted growth in "traditional" commercial aviation, creating modernization requirements for the National Airspace System (NAS) to meet
increasing needs for capacity with safety through such initiatives as NextGen. The recent introduction of unmanned systems into the NAS will bring further challenges and opportunities for the
nation's aviation system. The integration of such a complex transportation system creates a clear need to develop new technologies and innovative operational concepts for secure and safe assured autonomy.
An unmanned systems future will see the integration of a wide variety of Unmanned Aerial Systems (UAS), personal air vehicles, Urban Air Mobility (UAM) vehicles, and cargo and special mission aircraft into the NAS.
These developments can leverage UAS Traffic Management (UTM) advancements for the unique requirements of UAM airspace management.
This vision must include robust human-machine decision-making systems for increasingly
autonomous traffic management. These unmanned systems and the associated decision-making infrastructure systems must handle atmospheric, traffic, and airspace uncertainties and must operate securely and safely at different
levels of system complexity. Adopting this vision will provide greater diversity and richness for the NAS; however, critical technical challenges must be addressed to ensure secure and safe cyber-physical aviation systems.
The main challenges include: (i) sensing and understanding complex operational surroundings, coordinating different types of aerial vehicles, planning and navigation through highly dynamic and uncertain environments; (ii)
securing the NAS against a wide range of malicious adversarial threats, specifically cyber-physical attacks; (iii) verifying and validating autonomous system operation; and, finally, (iv) properly integrating new vehicles
and traffic management approaches in the midst of autonomy. Our primary goal is to ensure safe, secure and robust integration of autonomous vehicles into a UAM-tailored transportation infrastructure while maintaining
compliance with existing commercial and civil air transportation safety standards.
For more details see our published work.
Aerodynamic Intelligent Morphing System (A-IMS) for Autonomous Smart Utility Truck Safety and Productivity in Severe Environments
Funding Source: NSF S&AS-1849198,
Collaborators: V. Vantsevich and several others from the University of Alabama, Birmingham
Utility trucks are the first responders in areas of extreme climate and severe weather situations for recovering life beings from destroyed properties, for cutting trees to restore traffic, and for repairing electric posts and restoring power. It is thus crucial to create an advanced and effective scientific framework for maintaining productivity and safety while simultaneously eliminating accidents. Towards that direction, this project aims to establish a novel and integrated framework to monitor and predict weather conditions and feed such information into reinforcement learning mechanisms that will enable us to autonomously shape the utility trucks.
Unlike conventional approaches, the A-IMS will integrate model-free shape-morphing learning mechanisms with model-based interactive design to manage air-fluid flows, based on the road conditions, meteorology, speed limit, wind speed, and direction. This potentially transformative framework for A-IMS will: (1) bring new perspectives of learning to enhance smartness of natural-engineering systems that leverage physical and information processes; (2) establish an integrated design framework for hazardous environments to achieve resilience, and productivity through integrated adaptation of morphological properties while also mitigating the effects of potentially adversarial learning agents that can exist in the cloud; (3) investigate the interactive physical components of the A-IMS, that will simultaneously operate in two different mediums of multi-phase fluids, and solids (i.e., the air/fluid and road). The A-IMS framework shall be evaluated through, (i) hardware/software implementation, and in (ii) real-world conditions in the unique facilities of the Wall of Wind at the Florida International University.
For more details see our published work.
Non-Equilibrium Game-Theoretic Learning Techniques For Mitigating Adversaries in Complex Adaptive Systems (CAS)
Funding Source: ARO YIP
The networks and information systems that are being constructed today are extremely complex. A cyberphysical
attack against a network could have cascading and devastating effects on other portions of the information
enterprise. To defend against a network wide attack, it is imperative that we know what is on our
systems and their composition. We need to know what computers we have, what applications are running,
what vulnerabilities exist and what networks are related to other networks. Only when armed with this type of
information can we possibly adapt our countermeasures and protection procedures to counter an attack.
The objective of this project is to develop novel learning frameworks for non-equilibrium game-theoretic adaptive
algorithms for systems in adversarial, dynamic and uncertain environments. The technical approach is
to be inspired by interdisciplinary ideas from different fields, such as game theory, nonlinear adaptive learning
theory, cognitive science and network security. The anticipated outcomes include the development of
(i) level-k learning architectures for security, where decision makers are not perfectly rational and do not possess infinite
intelligence; (ii) a learning-based moving target defense that reduces predictability; and (iii) a resilient learning
framework to bandwidth depletion attacks. The contributions and impact on Army's objectives will be
the ability to multiply engagement capability, and enable coordination of distributed, heterogeneous teams of
manned, unmanned vehicles and humans in dynamic, adversarial and completely uncertain environments. The
models, theories, techniques, and tools developed as part of the proposed research will lay the foundation for
the DoD to develop more effective techniques against intelligent attackers with different levels of rationality and
For more details see our published work.
Intermittent Learning Framework for Smart and Efficient Cyber-Physical Autonomy
Funding Source: NSF CAREER CPS-1851588, NSF CAREER CPS-1750789,
Current learning algorithms cannot be easily applied in CPS due to their need for
continuous and expensive updates, with the current triggered frameworks having fundamental limitations.
Such limitations lead to the following questions. How can we incorporate and fully adapt to totally
unknown, dynamic, and uncertain environments? How do we co-design the action and the intermittent
schemes? How can we provide quantifiable real-time performance, stability and robustness guarantees by
design? And how do we solve congestion and guarantee security? We will build on our multiyear experience
and develop fundamental contributions to CPS by providing novel frameworks that will allow the
fully autonomous operation in the face of unknown, bandwidth restricted, and adversarial environments.
As specific merits, the project will, (i) unify new perspectives of learning in engineering to enable smart
autonomy, resiliency, bandwidth efficiency, robustness, real-time optimality and adaptation that cannot
be achieved with the state-of-the-art approaches; (ii) develop intermittent deep learning methods for CPS
that can mitigate sensor attacks by dynamically isolating the suspicious components and can handle cases
of limited sensing capabilities; (iii) incorporate nonequilibrium game-theoretic learning in CPS with
components that do not share similar mechanisms for decision making and do not have the same level
of rationality due to heterogeneity and may differ in either their information obtained or their ability in
utilizing it; and (iv) investigate ways to transfer intermittent learning experiences among the agents.
For more details see the project's webpage and our published work.
Data-Driven Learning Techniques for Cyber-Physical-Situation Awareness in Defense Systems
Funding Source: ONR Minerva Research Initiative
The objective of this project is to investigate and develop novel data-driven learning techniques for realtime
and adaptive cyber-physical situation awareness of adversarial activities, and feed information about
the impact of those activities back to the security analysts. This will significantly enhance awareness and
safety in defense systems by providing fast decision making along with, guarantees for mission completion
and conditions for robustness. The guaranteed robustness of the proposed data-driven learning techniques
will ensure fast prompt and/or automated decision making without reaching back to higher command levels.
The technical approach is to be inspired by interdisciplinary ideas from different fields, such as game theory,
cognitive learning (specifically reinforcement learning), cyber-physical systems, and network security.
The anticipated outcomes include transformative data-driven learning algorithms to engage the DoD security
analysts in the characterization, recognition and mitigation of adversarial actions, with the support of
rapid decision-making through prompt action, and recommendations/actions from "automated assistants." For more details see our published work.
Enforcement of Geofencing Policies for Commercial Unmanned Aircraft Systems
Funding Source: NSF SATC-1801611
Collaborators: R. Gerdes, M. Farhood, J. M. Park (Virginia Tech), M. Li (University of Arizona)
The adoption of services and applications that rely on Unmanned Aircraft Systems (UAS) is expected to continue its acceleration for the foreseeable future. Such services and applications include search and rescue, crop monitoring, real-time traffic monitoring, critical infrastructure inspections, and others. As UAS continue to proliferate, aviation regulatory agencies around the world are facing a challenging dilemma in terms of safely integrating UAS operations into their respective national airspace systems and enforcing their compliance to relevant regulations, including enforcement of no fly-zones. Current UAS lack effective on-board flight restrictions that would prevent them from entering critical airspace, e.g., airport runways and nuclear power plants. Furthermore, it is essential that the safety and privacy of members of the public be safeguarded from accidental and non-accidental UAS incursions. This project aims to address these critical needs by developing offensive and defensive measures for detecting, identifying, and bringing down UAS that violate controlled/restricted airspace in an automated, controlled, and reliable manner.
A three phase approach to thwarting a UAS encroaching on restricted airspace will be used: (1) detect the incursion using passive radar that exploits signals of opportunity and radio frequency (RF) communications/emissions from the encroacher itself; (2) deploy one or more defender UAS, capable of learning the motivations and behaviors of the encroacher, to pursue the UAS and gain and maintain proximity to it even should the encroacher take evasive action; and (3) have the defender UAS launch offensive measures that manipulate/compromise the sensor and actuation signals the encroacher relies on to stay airborne. For more details see our published work.
Enabling Secure and Resilient Extremely Fast Charging: A Software/Hardware-Security Co-Design Approach
Funding Source: Department of Energy
Collaborators: R. Gerdes and several others from Virginia Tech, Utah State, OnBoard Security, ChargePoint Inc., Ford Motor Co., and Commonwealth Edison Company
Extremely fast charging (XFC) has the potential to reduce the charging time of battery electric vehicles (BEV)
to be equivalent to the filling time of internal combustion engine vehicles (ICEV), thus eliminating one of the
few advantages ICEV still posses for light- and heavy-duty vehicles. Enabling XFC will, however, require
coordination and cooperation between the grid, charging stations, and the vehicles themselves, which leads
to an inevitable increase in the attack surface for all systems combined.
In securing the overall system,
we must not only embrace traditional cybersecurity, which is chiefly concerned with communications and
the operation of digital systems, but also cyber-physical systems security as the proper operation of XFC is
critically dependent on systems' abilities to know about (sense) and interact with (actuate) the physical world. For more details see our published work.
Mobility in Severe Terrain Environments
Funding Source: NATO SPS G5176
Collaborators: V. Vantsevich (University of Alabama)
The proposed project's effort is to develop and demonstrate a new technology to radically enhance vehicle tactical and operational mobility and survivability by establishing agile and interactive dynamics of innovative vehicle sub-systems. The scope of the proposed project is fundamental and is based on applied research in the multi-disciplinary mechatronics areas including ground vehicle engineering, sensor design and application, electronics and controls.
The new technology will enable vehicles of any weight class, with any number of wheels, independently driven wheels, or through a drive-train a unique ability to maintain mobility through high-speed and rapid maneuvers in complex off-road environments. Ramifications from this technology will greatly increase crew safety, reduce roll-overs, and enables vehicles too rapidly and aggressively veer/steer or experience external blast/force in on-road, off-road, and poor terrain conditions while maintaining vehicle control and mobility. The proposed system is a vehicle by-wire sub-system that controls and monitors the vehicle's mobility based on input programs of the motions via any autonomous controller or human operator input. For more details see our published work.
Security of Underwater Cyber-Physical Systems
Funding Source: NAVSEA
Collaborators: D. Stilwell, M. Psiaki (Virginia Tech)
The convergence of systems on modern UUVs and the reliance of the industry on connectivity leads
to a broad space for exploitable vulnerabilities. Unfortunately, due to current re-certification practices,
software and/or hardware fixes to address such vulnerabilities could take months to appear. The goal of
the proposed research is to increase the security of underwater cyber-physical systems (UCPS), which
include the UUVs themselves and necessary supporting infrastructure, through reductions in attack
surface, decreased time to patch, and increased resiliency to attacks, thus ensuring safety, integrity,
The overall objectives of this project are to assess cybersecurity risks for UUVs
and to propose practical countermeasures. For more details see our published work.
Reinforcement Learning for Optimal Adaptive Feedback Control and Games
Collaborators: F. L. Lewis (U. Texas), J. P. Hespanha (UCSB), W. Dixon (U. Florida), G. R. Hudas (US Army TARDEC), R. Babuska (TU Delft), D. Vrabie (PNNL), R. Kamalapurkar (Oklahoma State U.), S. Bhasin (IITD)
Adaptive control and optimal control represent different philosophies for
designing feedback control systems. These methods have been developed by the
Control Systems Community of engineers. Optimal controllers minimize user prescribed
performance functions and are normally designed offline by solving
Hamilton-Jacobi (HJ) design equations, for example, the Riccati equation,
using complete knowledge of the system dynamical model. However, it is often
difficult to determine an accurate dynamical model of practical systems.
We propose new techniques based on approximate dynamic programming that allow the design of adaptive
control systems with novel structures that learn the solutions to optimal control problems and Nash/Stackelberg games
in real time by observing data along the system trajectories. Complicated Lyapunov stability proofs ensure boundedness of the closed-loop signals. For more details see our published work.