Abstract

Autonomous ground vehicles (AGVs) operating collaboratively have several advantages over vehicles operating alone. An AGV team may be more resilient and efficient than a single AGV. Other advantages of AGV teams include increased coverage and multiple viewing angles of terrain features as well as resistance to failure from any single AGV. Additionally, AGV teams can explore large terrains more quickly and thoroughly than a single system. In this work, the feasibility of using a team of high-mobility AGV to explore a navigation corridor, map the terrain, and autonomously flag obstacles for future navigation is evaluated. Focusing on negative obstacles, the value of using multiple vehicles to map a navigation corridor is quantified. This study is the first to evaluate large teams of AGV collaborating in realistic off-road, 3D environments. The feasibility of the large-scale AGV team is demonstrated while avoiding the high cost of purchasing and testing large numbers of vehicles using the Mississippi State University autonomous vehicle simulator (MAVS), a high-fidelity, physics-based simulation tool. The cost and benefits of increasing the AGV team size are evaluated. The simulation results show how factors like fuel use, map coverage, and obstacle detection are influenced by increasing numbers of AGV in the team. The simulation architecture is presented and experiments quantifying the performance of the simulator are shown. Finally, a model for evaluating the tradeoff between mission effectiveness and fuel use is developed and presented to demonstrate the utility of this approach.

1 Introduction

The capability and availability of self-driving cars have increased rapidly in the last decade [1,2]. The recent capability improvements of self-driving cars have largely been enabled by the development of machine learning algorithms [3]. However, despite recent advances, many challenges remain for self-driving vehicles. This is particularly true for self-driving in off-road terrain, which presents many challenges not encountered in highway and urban driving [4]. These include the absence of clearly marked travel lanes and the presence of vegetation, soft soil, rough surfaces, and negative obstacles.

The ability of a self-driving vehicle to traverse off-road terrain strongly depends on the characteristics of the vehicle like its size, weight, and tire properties [5]. Successful navigation of off-road terrain by a self-driving autonomous ground vehicle (AGV) requires the vehicle to measure the properties of the terrain with on-board sensing and compare the terrain properties to the vehicle capabilities in real-time. Despite advances in machine learning algorithms, this process of terrain perception is often limited by the range, resolution, and field-of-view of the sensors on the AGV—and this is especially true in the case of negative obstacles [6].

One potential method for mitigating perception limitations imposed by sensor range and field-of-view is to employ multiple autonomous agents viewing the terrain from different angles [7]. A team of multiple AGV cooperating independently is known as a swarm. While swarming concepts have been tested at fairly large scales for unmanned aerial vehicles (UAVs) [8], little work has been done to evaluate the effectiveness of AGV swarms in off-road environments.

In this work, an AGV swarm system for terrain exploration is presented with a focus on off-road mobility considerations and negative obstacle detection. A physics-based simulator, the Mississippi State University autonomous vehicle simulator (MAVS) [9], is used to demonstrate the swarm concept. MAVS was chosen for this work for two important reasons. First, MAVS is compatible with high-performance computing (HPC). Given that the main goal of this work was to simulate large-scale teaming of AGV, HPC was a requirement that no other publicly available simulator could easily meet. Second, MAVS provides detailed off-road environments, including geometrically detailed representations of vegetation, that give realistic results for lidar simulation that far exceed other simulators [9]. Using MAVS, a series of simulated experiments are conducted to evaluate the ability of the swarm to detect negative obstacles. These results are compared to the overall fuel consumption of the swarm system, failure rates, and mission completion time, arriving at an optimal number of AGV for terrain exploration and negative obstacle detection. This work presents novel contributions in both the size of the swarm (up to 100 robots) and the realism of the off-road terrain considered, as will be further shown in the following sections.

2 Background and Related Work

Simulation has become a critical enabler to the development and testing of AGV [10]. Simulation can be used early in the development phase to find critical bugs and optimize design considerations like sensor placement [11]. In the testing phase, simulation allows a large number of repeatable scenarios to be tested [12]. When it comes to multi-vehicle systems and swarms, the need for simulation is even greater because of the cost and logistics associated with maintaining a large fleet of AGV.

Some of the earliest work on robot teaming was performed by Burgard et al., who published a study of multi-robot exploration with a team of two robots in an indoor space [13,14]. Their system was evaluated using a purpose-built top-down 2D simulator. Similar early work was published by Rekleitis et al. [15], who performed multi-robot exploration experiments with a focus on collaborative localization using the robodaemon software [16]. Additional work on collaborative, team-based mapping was performed around this time by several groups [1720].

Subsequent work developed methods for maintaining communication in distributed multi-robot networks [21], merging occupancy grids from teams of robots [22], and obeying constraints of wireless networks during exploration [23]. Research on optimizing exploration algorithms for robot teams also continued at this time [24,25].

Prior to the development and adoption of the robotic operating system (ROS) [26], much work was focused on developing interoperability between robots using custom-built simulators. For example, Brudnak et al. developed a real-time distributed simulator for up to 10 vehicles using high-level architecture [27]. Barnes et al. [28] developed a technique for controlling a group of small ground robots using potential fields and demonstrated the method experimentally. Saad et al. [29] developed a testbed for teams of up to a dozen small unmanned aerial vehicles (UAV) with a complementary 3D simulator based on a game engine.

Indoor exploration by a team of three robots was studied by Wurm et al. [25] using the Carnegie Mellon Robot Navigation Toolkit [30]. Later, Atyabi et al. [31] created a custom simulator, Magician, to study heterogeneous teams of robots operating in urban environments. Like most of the simulators developed in these studies, Magician used a two-dimensional, top-down representation of the environment with simplified vehicle dynamics and sensor models. The primary focus of these early works was studying the theoretical framework for employing multi-robot systems. Further work in this area was done by Senthilkumar and Bharadwaj [32] and later Jezdimirović et al. [33], who created a simplistic 2D simulation environment for coordinated attack movements for groups of three or four vehicles. In addition, Krajzewicz et al. developed control algorithms for lane change maneuvers in groups of autonomous vehicles and tested the algorithms with the urban mobility simulator, SUMO [34]. Additionally, some recent work has used 2D, matlab-based simulators to study vehicle control and rollover mitigation [35,36].

With the widespread adoption of ROS, the availability of high-quality game engines, and the advancement of computing systems, multi-robot simulations have become more sophisticated in recent years. For example, Feierle et al. [37] simulated an autonomous and manned vehicle negotiating a single lane intersection, while Barciśet al. [38] simulated swarms of small robots in ROS2.

Many simulators using game engine technology have progressed beyond the 2D, top-down approach used in the past. Nieto-Granda et al. [39] evaluated a group of nine turtlebots cooperating to explore an indoor environment using the Stage simulator [40], while military autonomous convoy and leader-follower scenarios with up to four vehicles were simulated by Brabbs et al. [41] using the ANVEL simulator [42]. Barlow and Garratt [43] simulated a group of eight small ground vehicles moving collectively using CoppeliaSim [44]. In addition, multiple vehicles can be simulated in CARLA [45] using OpenScenario [46]. Lee et al. have used this feature to simulate V2X architectures in urban and highway environments [47].

The availability of high-performance computing (HPC) has also enabled new advances in swarm simulations, both in terms of realism and scale of team size. For example, Goodin et al. [48] simulated a four-vehicle team of high mobility multipurpose wheeled vehicle (HMMWV)-sized vehicles moving in biologically inspired formations using the physics-based virtual autonomous navigation environment simulator [49]. In contrast, Kuru and Kahn [50] developed an agent-level simulator for evaluating a framework for a “smart-city” for AGV navigation with up to 14,000 vehicles operating in the city environment.

This work extends these recent advances using a physics-based simulator to perform high-fidelity, off-road simulations of the vehicle sensors and dynamics. The simulation is scaled to large numbers of vehicles by distributing the vehicle simulation processes on HPC to create a simulation environment that exceeds previous efforts in both scale and realism for off-road environments. This work is the first of its kind to study large (>20) teams of AGV operating in realistic off-road environments.

3 Simulated Autonomous System

The goal of the swarm system was to autonomously explore the area of operation and build a map of the obstacles in the mobility corridor. This was accomplished using a team of high-mobility light wheeled vehicles, composed of the Polaris MRZR-4 [51]. Each vehicle was equipped with a lidar sensor and localization sensor. The simulated lidar was a 64-beam Ouster OS2 [52] and the simulated localization sensor was a real-time kinematic corrected multi-band GPS receiver, the Swiftnav Piksi [53]. The lidar sensor was mounted on the front hood of the vehicle and tilted down at an angle of 25 deg in order to better scan the ground surface in front of the vehicle. The localization sensor was mounted on the roof of the vehicle, as shown in Fig. 1.

Each vehicle was equipped with its own autonomous control, capable of operating the vehicle independently of the swarm. The map collected by each vehicle was reported to the group through peer-to-peer communication. In the following subsections, the autonomy algorithms on each vehicle and the swarm control strategies are discussed in more detail.

The autonomy package for each vehicle consisted of three main modules: perception, path planning, and vehicle control. The perception module processed the lidar data to identify positive and negative obstacles. The path planning module selected a route for the vehicle from its current position to the goal point. And the control module calculated the appropriate throttle, steering, and braking commands to safely follow the desired path.

While many AGV use ROS to communicate data between the perception, planning, and control modules, ROS was not ideal for this application because of the relative difficulty of simultaneously operating dozens of individual systems in ROS, in addition to the difficulty of communicating between those systems with ROS. Instead, the MAVS simulator was embedded in a messaging architecture that used the message-passing interface (MPI) [54] to send messages between the AGV team agents.

3.1 Perception.

The perception module identified obstacles geometrically using a combination of estimated slope and curvature to identify potential obstacles. In the first step of the perception, the point cloud from each successive lidar scan was registered to a global, static map using the transform provided by the real-time kinematic (RTK) localization sensor. A voxel filter was applied to the points, placing them into a 2D grid. In each grid cell, only the highest and lowest points were retained. For each cell, an “in-cell” slope was calculated by dividing the elevation difference between the highest and lowest points by the dimension of the cell. Cells with slope above a certain threshold (2.5 in this case) were flagged as obstacles. This is similar to the approach used by Rankin et al. [55].

For negative obstacle detection, the surface curvature was calculated at each cell using a five-point stencil. Assuming that the lowest measured point in each cell was the ground surface elevation, h, then the curvature in cell (i, j) is given by [7]
(1)
If the absolute value of the curvature exceeded 0.5, then the cell was flagged as potential obstacle.

3.2 Path Planning and Vehicle Control.

The path planning module consisted of two stages. In the first stage, a global planner using A* [56,57] calculated a path through the map from the vehicles current position to the goal point. Since the terrains and maps were large, these paths could be several hundred meters long. In addition, the A* algorithm makes no consideration for the dynamics of the vehicle or the relative safety of the path. Therefore, in the second stage, a local planner calculated a near-field path that was feasible for the vehicle to follow. The local planner uses cubic splines and accounts for vehicle steering, collision avoidance, and minimizing steering changes [58], resulting in a smooth, drivable path.

The vehicle control module used the pure pursuit algorithm [59] to calculate steering based on the desired path output from the local planner. The throttle setting was calculated using a PID controller [60], which attempted to maintain a speed of 5 m/s for the vehicle. Throttle, steering, and braking commands were passed to MAVS using an array of three floats.

3.3 Swarm Control.

The swarm control model used in this work utilized a central authority to merge and manage the global map and assign goal points for each robot. In the initialization step, the central authority assigned a goal point to each agent. The goal points were distributed to ensure maximum exploration of the mobility corridor. Each agent independently and autonomously navigated to its goal point. At regular intervals, the robots sent their current map back to the central authority. The central authority generated the most up-to-date global map representation by combining the agents’ local representations and ensured that all agents had the current global representation by rebroadcasting the combined map. The global map was updated by the central authority at a rate of 10 Hz. Inter-AGV communication was enabled by MPI messaging with an array of floats for the state of each vehicle.

The maps were represented by a 2D occupancy grid. Figure 2 shows typical occupancy grids from the simulation, as well as the evolution of the map completeness over the course of a single mission. The top figure shows the map at the beginning of the experiment, while the bottom one shows the map near the end of the experiment.

4 Simulation Environment

The simulation environment used for this study was based on the MAVS software. The MAVS was integrated into an HPC-based distributed architecture that allowed dozens of vehicles to be simulated simultaneously. The following sections discuss the simulator and distributed simulation environment in more detail.

4.1 Mississippi State University Autonomous Vehicle Simulator.

The MAVS was used to simulate the vehicles navigating in the off-road environments. The MAVS lidar simulation has been validated for use in dense vegetation [9] and rain [61]. The MAVS has also been used in a variety of studies related to lidar performance in autonomy including machine learning algorithms for classifying off-road terrain [62] and for detecting obstacles in dense vegetation [63]. The MAVS uses the Embree ray-tracing kernel [64] to accurately account for effects like lidar beam size, surface reflectance properties, and low-angle reflectance in lidar returns. For the RTK localization sensor simulation, MAVS uses an empirical model based on field measurements [53].

The vehicle suspension is simulated using a lumped parameter, independent linear spring damper model for each suspension element. This is similar to the approach used in Vehdyn [65], but extended to 3D using the Reactphysic3D multibody-dynamics library [66]. The tire is simulated with a radial-spring model [67] with the model extended to 3D by dividing the tire into multiple 2D slices. The tire–soil interaction is simulated with empirical models derived from the DROVE database for sand [68] and clay [69]. More information on the MAVS vehicle model can be found in recent works by Moore et al. [70] and Islam et al. [71].

4.2 Communication Model.

To simulate the ad hoc communications of the simulated vehicles, a simple model consisting of binomial probabilistic links and a many-to-many propagation strategy is used [72]. Using a distance-based probability function, the probability of successful communication between every pair of vehicles is computed. These probabilities are then used to share vehicle data only via successful, direct links between vehicles. Over time, communication data will propagate throughout the vehicle network, dependent on the existing communication links.

The distance-based probability function used in the simulated communication model is a logistic sigmoid function designed by selecting a desired failure rate at the nominal range and the failure rate at twice the nominal range. In this model, the relative signal strength between robot i and robot j, σij, is given by
(2)
where rij is the distance between the two robots in meters. The signal gain (σg) and the signal bias (σb) are empirical parameters given by
(3)
and
(4)
where rmax is the nominal maximum communication range, fr is the failure rate at rmax, and fr2 is the failure rate at 2 × rmax. In the experiments shown in the following sections, the communication range was taken to be rmax = 100.0, the failure rate was fr = 0.001, and the double range failure rate was fr2 = 0.5. This resulted in a communication model with parameters of σg ≈ 0.0691068 and σb ≈ −13.8135. The resulting model is shown in Fig. 3. Connectivity is calculated by comparing the signal strength to a uniform random test variable on the range [0,1].

This model allows coarse designing of a probability function with three major sections: a flat, high connectivity region within the nominal range followed by a region with gradually decreasing connection strength and a final, low connectivity region. While phenomenologically based, the posing of parameters in terms of desired behavior, ranges, and failure rates allows tuning of the model to capture specific network conditions irrespective of the broader, more complex environmental conditions that would predicate such conditions.

4.3 Distributed Simulation.

The simulation was distributed using a custom-designed hybrid memory model that used a shared memory approach, implemented with OpenMP [73], to simulate the individual sensors and vehicles and a distributed model, implemented in MPI [54], to facilitate communication between each vehicle.

The first step in the simulation loop is to update the state of each agent (AGV). This includes updating driving commands using the on-board autonomy, updating the dynamic state of the vehicle platform using those driving commands, and updating the sensors on the AGV. The vehicles are updated in a parallelized for-loop, meaning that all the vehicles can be updated simultaneously in parallel.

Following the vehicle update loop, the updated pose (position and orientation) of each vehicle is broadcast to all other vehicles using the MPI_Allreduce function. The AGV-to-AGV communication model presented in Sec. 4.2 is then updated with the revised vehicle positions.

In the next step, the occupancy grid of each agent is broadcast to all other agents using the MPI_Allreduce function. We note that this is not simulating a physical process—rather, the communication model is then used to calculate the transmission of each grid between robots to the central authority.

In the final step, the central authority checks to see if all agents have reached their final goal. If they have not, the central authority uses the grids it received to create a combined occupancy grid which is rebroadcast to all agents within the communication network. This loop repeats until the simulation is complete.

5 Exploration Scenario

A reconnaissance scenario for an avenue of approach in a military context was constructed in order to evaluate the efficacy of the swarm concept. The goal of the reconnaissance is to select a mobility corridor that supports maneuver. According to Weigeshoff [74]:

Avenues of approach are routes by which a force may reach key terrain or an objective. Avenues of approach are evaluated according to their maneuver support potential, access to key terrain or adjacent avenues of approach, degree of canalization, cover and concealment, observation and fields of fire, and obstacles. Mobility corridors are areas which permit movement and maneuver. They permit friendly and enemy forces to advance or withdraw in doctrinal configurations, and to capitalize on the principals of mass, shock, momentum, and speed.

Therefore, in this scenario, an avenue of approach that featured rolling terrain marked by both positive (trees) and negative (holes) obstacles was constructed. A new terrain was randomly generated for each trial with holes and trees at random locations. An example of one scene used in the simulated experiments is shown in Fig. 4. By completing many different trials of the experiment, each one in a randomly generated scene, the performance variables can be treated statistically rather than deterministically.

The surface mesh resolution was 0.25 m. The negative obstacles had a depth of 1.5 m, a radius of 2.5 m, and a parabolic shape. The vegetation consisted of sparse, mature trees of species consistent with the southeastern United States including oaks, pines, pecans, and magnolias.

To start the scenario, the vehicles were distributed in row across the western edge of the map. The vehicles were uniformly spaced in the north–south direction. Each vehicle attempted to navigate to a goal point at the eastern edge of the map while maintaining a speed of 5 m/s.

5.1 Independent Variable and Performance Metrics.

The number of robots (NR) was the independent variable in this analysis. The vegetation density and surface roughness were varied randomly within a small range to introduce to randomness to the individual simulations. The number of robots was varied between 1 and 24, while the vegetation density was varied between no trees and approximately 10 trees/km2.

For each trial, there were five dependent variables (metrics).

  1. Time to complete (TTC)—the number of seconds it takes for all AGV to arrive at their goal point to complete the mission.

  2. Failure rate (δf)—the fraction of trials that all AGV do not reach the goal point in the allotted mission time.

  3. Fuel usage (fv)—the total fuel usage for the AGV system.

  4. Negative obstacle detection rate (δd)—the fraction of negative obstacles detected.

  5. Fraction of terrain explored (δt).

The TTC metric is straightforward to measure—it is simply the amount of time in seconds it took for all vehicles to reach their goal.

For the failure rate metric, a trial was considered a “success” if all vehicles reached the goal point in the allotted time; otherwise, the mission was deemed a failure. The failure rate was the number of failures divided by the number of total trials for a given configuration. Therefore, the optimal failure rate is 0.0.

Negative obstacle detection rate was determined by counting the number of negative obstacles detected by the AGV team and dividing this by the total number of negative obstacles in the terrain. Therefore, the optimal detection rate was 1.0.

Fuel consumption depends on many factors like air temperature, tire inflation, and engine properties. While modeling the fuel consumption for the MRZR4 is beyond the scope of this work, Lee et al. [75] determined that fuel consumption is linearly related to throttle position. Therefore, for comparative purposes, the cumulative throttle position for all vehicles over the course of the mission was used as a proxy for the fuel consumption metric.

The fraction of the terrain explored is related to the occupancy grid. A grid cell is considered “explored” if there are at least three lidar returns within that cell. The fraction of the terrain explored is the total number of explored cells divided by the total number of cells in the map.

5.2 Statistical Techniques.

The following subsection outlines the statistical techniques used in this work. While many analyses use statistical techniques appropriate for normally distributed data, much past work has shown that many performance metrics for ground vehicles are not normally distributed but rather follow binomial statistics or lognormal statistics. For convenience and clarity, these distributions and their application to AGV analysis are discussed in more detail in the following sections.

5.2.1 Failure Rate.

Pass/fail tests can only have two outcomes, therefore they follow binomial statistics. Examples of pass/fail problems in autonomy may include pedestrian detection [76] or obstacle avoidance [77]. While there are a number of ways to estimate the confidence interval (CI) using binomial statistics, for tests with low numbers of measurements or very low failure rates, a good approximation for the probability of success, p, and its uncertainty come from the Wilson score interval.
(5)
where n is the total number of trials, ns is the number of measured successes, nf = nns is the number of failures, and z is the (1 − α/2) quantile of a normal distribution. For a desired confidence interval, CI,
(6)
The desired quantile is then given by the cumulative distribution function of a zero-mean normal distribution at 1 − α/2.
(7)
where Φ is the probit of the 1 − α/2 quantile.

5.2.2 Time to Complete.

When conducting multiple tests and compiling TTC statistics, it is often illustrative to simply calculate the mean and variance of the TTC results. However, it is important to note that TTC do not follow the normal distribution. Rather, TTC are best fit by a lognormal distribution and the mean and confidence intervals should be calculated accordingly [78].

The lognormal distribution is the “statistical realization of the multiplicative product of many independent random variables, each of which is positive” [79]. The lognormal distribution is appropriate for positively skewed variables that can increase without limits but cannot fall below zero. The probability distribution function of the lognormal distribution is given by
(8)
For a set of TTC measurements, calculating the mean (μ′) and standard deviation (σ′) of the lognormal distribution is straightforward. However, it is important to note that in general, these are not the same as the parameters in Eq. (8). That is, μ′ ≠ μ and σ′ ≠ σ. The expected value for n measurements is simply the average of the measurements.
(9)
The variance is given by the deviation from the mean
(10)
From the mean and variance, the parameters of the lognormal distribution can be calculated.
(11)
and
(12)
Finally, from the lognormal parameters, the mean and variance are
(13)
and
(14)
Confidence intervals on μ′ should be calculated in the log space and finally converted back to the data space. This means that the error bars will be unequal on the upper and lower sides of the mean. For a given confidence interval, the upper and lower bounds are given by [80,81]
(15)
where z is the z-score from Eq. (7). Note that the μ and σ from Eq. (15) are the mean and standard deviation of the data in the logarithm-transformed space. To calculate the upper and lower bounds in the data space, the values must be transformed back into the data space.
(16)

6 Results

In this section, the results of two sets of simulated experiments are presented. In the first experiment, groups of 1–6 robots cooperatively explored a scene of 250 × 250 m2 with 25 randomly placed negative obstacles. The vegetation density and surface roughness were varied randomly. A total of 100 trials were run for each number of robots for a total of 500 simulated experiments.

In the second experiment, groups of 1–24 robots cooperatively explored a scene of 500 × 500 m2 with 100 randomly placed negative obstacles. Both the vegetation density and surface roughness were also randomly varied. A total of 100 trials each were run for 1, 2, 3, 4, 6, 8, 10, and 24 robots, for a total of 800 simulated experiments. It should be noted that the density of negative obstacles was the same for both experiments 1 and 2 (4 × 10−4 obstacles/m2), so the detection results are directly comparable.

The simulations were run on an Dell workstation with six Intel Xeon processors having 16 logical threads per processor for a total of 96 available threads. The simulations were run with eight threads per AGV. The 1400 simulations took a total of two weeks to complete. Results were saved to text files for each simulation, then compiled into a csv file for further analysis with python.

6.1 Experiment 1.

The results for experiment 1 are presented in this section. The results are summarized in Table 1. Negative obstacle detection rates, like failure rates, follow binomial statistics because each obstacle is either detected or not detected. Detection rates for each input configuration were analyzed using binomial statistics, and the results are plotted in Fig. 5.

Fuel consumption, like TTC, is a “statistical realization of the multiplicative product of many independent random variables, each of which is positive” [79]. Therefore, the fuel usage results are also analyzed using lognormal statistics. The results of the fuel usage for each configuration are plotted in Fig. 6.

6.2 Experiment 2.

Experiment 2 featured a larger terrain than experiment 1 (0.25 km2 versus 0.625 km2) and more autonomous agents (up to 24 for experiment 2 versus only six for experiment 1). The purpose of this additional experiment was to determine if the size of the scene had an effect on the observed metrics.

The results for experiment 2 are summarized in Table 2. Detection rates for experiment 2 are plotted in Fig. 7. The results of the fuel usage for experiment 2 are plotted in Fig. 8.

6.3 Simulation Scaling Tests.

One measure of performance of the overall simulation architecture is the scaling as more vehicles are added. Scaling tests were performed on Mississippi State University’s Orion high-performance computer. Orion is a Dell/EMC PowerEdge C6420 cluster composed of 1800 compute nodes, each with dual 20-core Xeon Gold 6148 processors (with a base processor frequency of 2.4 GHz) and a total of 345 TB of RAM. Eight of these nodes are large-memory nodes with 384 GB of RAM with the remaining 1792 nodes containing 192 GB of RAM. Each node is interconnected via Mellanox HDR100 InfiniBand (100 Gb/s).

Simulations were run on Orion’s regular nodes with 192 GB of RAM. The scene size was 600 × 1200 m. One goal was to see how large of a simulation could be reasonably run on Orion. The normal queueing system on Orion permits up to 10 nodes per job and up to 48 h of run time. The number of MPI processes per node (with one vehicle per MPI process) was limited primarily by the memory usage per process, so as the number of vehicles per simulation went up, additional nodes were added as necessary to provide the additional memory. Also it was insured that the number of processes was always less than or equal to the number of CPU cores. There were six simulations run with the resulting times, number of vehicles, and number of nodes used shown in Fig. 9.

The scaling results indicate that the communication architecture used between the vehicles does not scale well. Each vehicle had a copy of every other vehicles map in memory, and the communication simulation was used to evaluate how those maps were propagated through the communication network. This led to a drastic increase in the time for the MPI_Allreduce calls for the vehicle maps as the number of vehicles increased.

Additional simulations were run with a fixed number of vehicles, 32 in this case, on the big memory nodes (384 GB RAM) while increasing the number of nodes. Increasing the number of nodes adds more cpu cores and more memory but would likely add latency in the communication inter-node compared to intra-node. This is also due to the large amount of shared map data being stored in memory in this particular simulation architecture. Figure 10 shows the results.

7 Discussion

Taken together, experiments 1 and 2 reveal several interesting trends. The general trends show that as the number of robots increases, the amount of fuel used also increases and the fraction of obstacles detected increases. The results also show that the assumption of lognormal TTC statistics is reasonable.

In order to compare results from the two experiments, a new independent variable is defined by scaling NR by a power of the course length in meters, l,
(17)

The best value of α for each dependent variable is found through fitting the linearized data. The results of these analyses are presented in more detail in the following sections. In the following subsections, it will be shown that this approach can be used to produce an analytical relationship between the probability of negative obstacle detection and overall fuel usage for a swarm of AGV. This approach for deriving the mathematical relationship will be generally applicable for all swarm-based systems and presents a method for evaluating one relative tradeoff for AGV teaming—that is, the value of adding more robots to a team versus the relative fuel cost incurred by adding more AGV. Tradeoff models of this nature are essential for optimizing complex systems like AGV teams.

7.1 Model for Detection Rate.

One way to interpret the observed trend for the negative obstacle detection rate is to compare to the “expected” rate based on the predictions of an analytical model. For many obstacles, the probability of detection by a particular sensor, pd, is a function of the range (R), that is, pd(R).

According to Ref. [82], a good predictor for the detection range of negative obstacles by lidar is θov, where Δv is the vertical resolution of the lidar and θo is the angle between successive scans on the rear wall of the obstacle, given by
(18)
where H is the sensor height, w is the diameter of the negative obstacle, and R is the range to the obstacle. In the simulated experiments in this work, the lidar sensor had Δv ≈ 0.0062. The lidar sensor height was H ≈ 3.35, and the negative obstacle width was w = 2.5.
In order to model the detection rate from pd(R), the expected range between the target and the sensor needs to be estimated—given that the location of the target is not known a priori. It is well known that for a square of sides L, the expected value of the distance between two randomly chosen points is ≈0.5214L [83]. From Monte-Carlo analysis, it is found that in two dimensions, as the number of randomly emplaced sensors, N increases, the average distance from the target to any sensor decreases.
(19)
where ρs = N/L2 is the vehicle number density per square meter.
By substituting Eq. (19) into Eq. (18), a relationship between θov and the density of agents in the area can be attained. When the measured detection rate from both experiments is plotted against this predictor, a linear relationship of
(20)
is found. Figure 11 shows that this model reasonably collapses the data from experiments 1 and 2 into a single linear relationship. This linear relationship may be used to predict the expected detection rate, or as a way to interpret detection rates by swarms of vehicles.

7.2 Fuel Use and Detection Rate.

For analyzing the fuel usage from the combined data from experiments 1 and 2, it is found that αfv=1 yields the best fit. The results are plotted in Fig. 12.

With αfv=1, the resulting predictive equation for fuel usage is
(21)

For the negative obstacle detection rate, it is found that the combined data are best fit by αδd=1.55. The results of this fit for the combined data are shown in Fig. 13.

The best fit equation for the combined data from experiments 1 and 2 is
(22)
With Eqs. (21) and (22), the relative tradeoff between fuel use and negative obstacle detection for a given terrain can be calculated for the swarm. The detection rate per unit fuel (δd/fv) can be found by dividing Eq. (22) by Eq. (21)
(23)

This model is specific to this particular AGV team and detection tasks. However, the basic steps followed to derive this model from the data are applicable to any AGV swarm. These steps include (1) developing a linearized model for detection rate of the target of interest which relates the AGV team areal density to the probability of detection for the system, (2) developing a model for fuel usage as vehicles are added to the swarm, and (3) relating these two models based on the intended problem size to develop a prediction for the performance tradeoff, as presented in Eq. (23).

7.3 Validating Statistical Assumptions.

In Sec. 5.2.2, it was stated that TTC statistics follow a lognormal distribution, as opposed to a normal distribution. While many works mistakenly use normal statistics to analyze results, in this section it is shown why proper statistical treatment is necessary. Figure 14 shows the probability distribution of finish times in the 10 vehicle scenario of experiment 2. This subset of data features 98 TTC points, excluding the two failed tests which did not complete the scenario.

Figure 14 compares the fit to the data using lognormal statistics (solid line) versus normal statistics (dashed line). Clearly, the lognormal statistics used in this work fit the observations better, reproducing the asymmetrical tail of the observed distribution better than the symmetric normal distribution. This observation is typical for TTC data where the symmetric normal distribution overestimates the number of observations below the mean and underestimates the observations above the mean. Noting that the other measurements of experiments 1 and 2 also fit this trend, the choice of lognormal statistics is validated by observation.

8 Summary and Conclusions

In this work, a simulation framework for testing of large-scale, off-road swarms of passenger-sized of autonomous vehicles was presented. This framework presented several key developments, including the use of a high-fidelity, physics-based simulation tool to analyze swarm robotics in realistic off-road terrain, the utilization of distributed simulation, and the development of appropriate metrics and statistical analysis techniques for swarm robotics.

By using negative obstacle detection as a performance metric, a clear relationship between performance and fuel use was found as a function of the number of vehicles in the swarm. Additionally, the statistical approach for evaluating performance was validated by the observations. This work was the first of its kind to study large (>20) numbers of AGV operating collaboratively in realistic off-road terrain.

Future will focus on developing more advanced coordination algorithms for the swarm, including biologically inspired algorithms. The MPI communication architecture for shared map data will be improved to improve scaling, and additional AGV performance metrics will also be evaluated.

Acknowledgment

The research described and the resulting data presented herein, unless otherwise noted, were funded under PE 0602784A, Project T53 “Military Engineering Applied Research,” Task 11.3 under Contract No. W56HZV-17-C-0095, managed by the U.S. Army Combat Capabilities Development Command (CCDC) and the Engineer Research and Development Center (ERDC). The work described in this document was conducted at the Center for Advanced Vehicular Systems, Mississippi State University. Permission was granted by ERDC to publish this information.

Conflict of Interest

There are no conflicts of interest.

Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

References

1.
Daily
,
M.
,
Medasani
,
S.
,
Behringer
,
R.
, and
Trivedi
,
M.
,
2017
, “
Self-Driving Cars
,”
Computer
,
50
(
12
), pp.
18
23
.
2.
Badue
,
C.
,
Guidolini
,
R.
,
Carneiro
,
R. V.
,
Azevedo
,
P.
,
Cardoso
,
V. B.
,
Forechi
,
A.
,
Jesus
,
L.
,
Berriel
,
R.
,
Paixão
,
T. M.
, and
Mutz
,
F.
,
2020
, “
Self-Driving Cars: A Survey
,”
Exp. Syst. Appl.
,
165
, p.
113816
.
3.
Rao
,
Q.
, and
Frtunikj
,
J.
,
2018
, “
Deep Learning for Self-Driving Cars: Chances and Challenges
,”
Proceedings of the 1st International Workshop on Software Engineering for AI in Autonomous Systems
,
Gotenburg, Sweden
,
May 28
, pp.
35
38
.
4.
Kelly
,
A.
,
Stentz
,
A.
,
Amidi
,
O.
,
Bode
,
M.
,
Bradley
,
D.
,
Diaz-Calderon
,
A.
,
Happold
,
M.
,
Herman
,
H.
,
Mandelbaum
,
R.
, and
Pilarski
,
T.
,
2006
, “
Toward Reliable Off Road Autonomous Vehicles Operating in Challenging Environments
,”
Int. J. Rob. Res.
,
25
(
5–6
), pp.
449
483
.
5.
Goodin
,
C.
,
Dabbiru
,
L.
,
Hudson
,
C.
,
Mason
,
G.
,
Carruth
,
D.
, and
Doude
,
M.
,
2021
, “
Fast Terrain Traversability Estimation With Terrestrial Lidar in Off-Road Autonomous Navigation
,”
Unmanned Systems Technology XXIII
,
Orlando, FL
,
Apr. 12
, Vol.
11758
,
International Society for Optics and Photonics
, p.
117580O
.
6.
Matthies
,
L.
, and
Rankin
,
A.
,
2003
, “
Negative Obstacle Detection by Thermal Signature
,”
Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No. 03CH37453)
,
Las Vegas, NV
,
Oct. 27
, Vol.
1
,
IEEE
, pp.
906
913
.
7.
Goodin
,
C.
,
Carrillo
,
J.
,
Monroe
,
J. G.
,
Carruth
,
D. W.
, and
Hudson
,
C. R.
,
2021
, “
An Analytic Model for Negative Obstacle Detection With Lidar and Numerical Validation Using Physics-Based Simulation
,”
Sensors
,
21
(
9
), p.
3211
.
8.
Arnold
,
R.
,
Carey
,
K.
,
Abruzzo
,
B.
, and
Korpela
,
C.
,
2019
, “
What Is a Robot Swarm: A Definition for Swarming Robotics
,”
2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON)
,
New York, NY
,
Oct. 10
,
IEEE
, pp.
0074
0081
.
9.
Goodin
,
C.
,
Doude
,
M.
,
Hudson
,
C.
, and
Carruth
,
D.
,
2018
, “
Enabling Off-Road Autonomous Navigation-Simulation of Lidar in Dense Vegetation
,”
Electronics
,
7
(
9
), p.
154
.
10.
Huang
,
W.
,
Wang
,
K.
,
Lv
,
Y.
, and
Zhu
,
F.
,
2016
, “
Autonomous Vehicles Testing Methods Review
,”
2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC)
,
Rio de Janeiro, Brazil
,
Nov. 1
,
IEEE
, pp.
163
168
.
11.
Meadows
,
W.
,
Hudson
,
C.
,
Goodin
,
C.
,
Dabbiru
,
L.
,
Powell
,
B.
,
Doude
,
M.
,
Carruth
,
D.
,
Islam
,
M.
,
Ball
,
J. E.
, and
Tang
,
B.
,
2019
, “
Multi-Lidar Placement, Calibration, Co-registration, and Processing on a Subaru Forester for Off-Road Autonomous Vehicles Operations
,”
Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2019
,
Baltimore, MD
,
May 2
, Vol.
11009
,
International Society for Optics and Photonics
, p.
110090J
.
12.
Schöner
,
H.-P.
,
2018
, “
Simulation in Development and Testing of Autonomous Vehicles
,”
18th Stuttgart International Symposium
,
Stuttgart, Germany Mar. 15
, pp.
3
14
.
13.
Burgard
,
W.
,
Moors
,
M.
,
Fox
,
D.
,
Simmons
,
R.
, and
Thrun
,
S.
,
2000
, “
Collaborative Multi-Robot Exploration
,”
Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065)
,
San Francisco, CA
,
Apr. 24
, Vol.
1
,
IEEE
, pp.
476
481
.
14.
Burgard
,
W.
,
Moors
,
M.
,
Stachniss
,
C.
, and
Schneider
,
F. E.
,
2005
, “
Coordinated Multi-Robot Exploration
,”
IEEE Trans. Robot.
,
21
(
3
), pp.
376
386
.
15.
Rekleitis
,
I.
,
Dudek
,
G.
, and
Milios
,
E.
,
2001
, “
Multi-Robot Collaboration for Robust Exploration
,”
Ann. Math. Artif. Intell.
,
31
(
1
), pp.
7
40
.
16.
Dudek
,
G.
, and
Sim
,
R.
,
2003
, “
Robodaemon—A Device Independent, Network-Oriented, Modular Mobile Robot Controller
,”
2003 IEEE International Conference on Robotics and Automation (Cat. No. 03CH37422)
,
Taipei, Taiwan
,
Sept. 14
, Vol.
3
,
IEEE
, pp.
3434
3440
.
17.
Simmons
,
R.
,
Apfelbaum
,
D.
,
Burgard
,
W.
,
Fox
,
D.
,
Moors
,
M.
,
Thrun
,
S.
, and
Younes
,
H.
,
2000
, “
Coordination for Multi-Robot Exploration and Mapping
,”
AAAI/IAAI
,
Austin, TX
,
Aug. 1
, pp.
852
858
18.
Thrun
,
S.
,
Burgard
,
W.
, and
Fox
,
D.
,
2000
, “
A Real-Time Algorithm for Mobile Robot Mapping With Applications to Multi-Robot and 3d Mapping
,”
Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065)
,
San Francisco, CA
,
Apr. 24
, Vol.
1
,
IEEE
, pp.
321
328
.
19.
Ko
,
J.
,
Stewart
,
B.
,
Fox
,
D.
,
Konolige
,
K.
, and
Limketkai
,
B.
,
2003
, “
A Practical, Decision-Theoretic Approach to Multi-Robot Mapping and Exploration
,”
Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No. 03CH37453)
,
Las Vegas, NV
,
Oct. 27
, Vol.
4
,
IEEE
, pp.
3232
3238
.
20.
Howard
,
A.
,
2004
, “
Multi-Robot Mapping Using Manifold Representations
,”
IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004
,
New Orleans, LA
,
Apr. 26
, Vol.
4
,
IEEE
, pp.
4198
4203
.
21.
Vazquez
,
J.
, and
Malcolm
,
C.
,
2004
, “
Distributed Multirobot Exploration Maintaining a Mobile Network
,”
2004 2nd International IEEE Conference on ‘Intelligent Systems’. Proceedings (IEEE Cat. No. 04EX791)
,
Varna, Bulgaria
,
June 22
, Vol.
3
,
IEEE
, pp.
113
118
.
22.
Birk
,
A.
, and
Carpin
,
S.
,
2006
, “
Merging Occupancy Grid Maps From Multiple Robots
,”
Proc. IEEE
,
94
(
7
), pp.
1384
1397
.
23.
Rooker
,
M. N.
, and
Birk
,
A.
,
2007
, “
Multi-Robot Exploration Under the Constraints of Wireless Networking
,”
Control Eng. Pract.
,
15
(
4
), pp.
435
445
.
24.
Fox
,
D.
,
Ko
,
J.
,
Konolige
,
K.
,
Limketkai
,
B.
,
Schulz
,
D.
, and
Stewart
,
B.
,
2006
, “
Distributed Multirobot Exploration and Mapping
,”
Proc. IEEE
,
94
(
7
), pp.
1325
1339
.
25.
Wurm
,
K. M.
,
Stachniss
,
C.
, and
Burgard
,
W.
,
2008
, “
Coordinated Multi-Robot Exploration Using a Segmentation of the Environment
,”
2008 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Nice, France
,
Sept. 22
,
IEEE
, pp.
1160
1165
.
26.
Quigley
,
M.
,
Conley
,
K.
,
Gerkey
,
B.
,
Faust
,
J.
,
Foote
,
T.
,
Leibs
,
J.
,
Wheeler
,
R.
, and
Ng
,
A. Y.
,
2009
, “
Ros: An Open-Source Robot Operating System
,”
ICRA Workshop on Open Source Software
, Vol.
3
,
Kobe, Japan
,
May 12
, p.
5
.
27.
Brudnak
,
M.
,
Nunez
,
P.
, and
Reid
,
A.
,
2002
, “
Real-Time, Distributed, Unmanned Ground Vehicle Dynamics and Mobility Simulation
,”
SAE International
.
28.
Barnes
,
L.
,
Fields
,
M.
, and
Valavanis
,
K.
,
2007
, “
Unmanned Ground Vehicle Swarm Formation Control Using Potential Fields
,”
2007 Mediterranean Conference on Control & Automation
,
Athens, Greece
,
June 27
,
IEEE
, pp.
1
8
.
29.
Saad
,
E.
,
Vian
,
J.
,
Clark
,
G.
, and
Bieniawski
,
S.
,
2009
, “
Vehicle Swarm Rapid Prototyping Testbed
,”
AIAA Infotech Aerospace Conference and AIAA Unmanned ·· · Unlimited Conference
,
Seattle, WA
,
July 21
, p.
1824
.
30.
Montemerlo
,
M.
,
Roy
,
N.
, and
Thrun
,
S.
,
2003
, “
Perspectives on Standardization in Mobile Robot Programming: The Carnegie Mellon Navigation (Carmen) Toolkit
,”
Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No. 03CH37453)
,
Las Vegas, NV
,
Oct. 27
, Vol.
3
,
IEEE
, pp.
2436
2441
.
31.
Atyabi
,
A.
,
Anderson
,
T. A.
,
Treharne
,
K.
, and
Powers
,
D. M.
,
2010
, “
Magician Simulator
,”
11th International Conference on Control, Automation, Robotics and Vision (ICARCV 2010)
,
Singapore
,
Dec. 7
, pp.
1
16
.
32.
Senthilkumar
,
K.
, and
Bharadwaj
,
K. K.
,
2012
, “
Multi-Robot Exploration and Terrain Coverage in an Unknown Environment
,”
Rob. Auton. Syst.
,
60
(
1
), pp.
123
132
.
33.
Jezdimirović
,
M.
,
Milinović
,
M.
,
Janković
,
R.
,
Jeremić
,
O.
, and
Pavić
,
M.
,
2013
, “
Basic Mathematical Model and Simplified Computer Simulation of Swarming Tactics for Unmanned Ground Combat Platforms
,”
Sci. Tech. Rev.
,
63
(
1
), pp.
17
24
.
34.
Krajzewicz
,
D.
,
Erdmann
,
J.
,
Behrisch
,
M.
, and
Bieker
,
L.
,
2012
, “
Recent Development and Applications of Sumo-Simulation of Urban Mobility
,”
Int. J. Adv. Syst. Measure.
,
5
(
3–4
), pp.
128
138
.
35.
Shao
,
K.
,
Zheng
,
J.
, and
Huang
,
K.
,
2019
, “
Robust Active Steering Control for Vehicle Rollover Prevention
,”
Int. J. Model. Identif. Control
,
32
(
1
), pp.
70
84
.
36.
Yan
,
C.
,
Shao
,
K.
,
Wang
,
X.
,
Zheng
,
J.
, and
Liang
,
B.
,
2021
, “
Reference Governor-Based Control for Active Rollover Avoidance of Mobile Robots
,”
2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
,
Melbourne, Australia
,
Oct. 17
,
IEEE
, pp.
429
435
.
37.
Feierle
,
A.
,
Rettenmaier
,
M.
,
Zeitlmeir
,
F.
, and
Bengler
,
K.
,
2020
, “
Multi-Vehicle Simulation in Urban Automated Driving: Technical Implementation and Added Benefit
,”
Information
,
11
(
5
), p.
272
.
38.
Barciś
,
A.
,
Barciś
,
M.
, and
Bettstetter
,
C.
,
2019
, “
Robots That Sync and Swarm: A Proof of Concept in Ros 2
,”
2019 International Symposium on Multi-Robot and Multi-Agent Systems (MRS)
,
New Brunswick, NJ
,
Aug. 22
,
IEEE
, pp.
98
104
.
39.
Nieto-Granda
,
C.
,
Rogers III
,
J. G.
, and
Christensen
,
H. I.
,
2014
, “
Coordination Strategies for Multi-Robot Exploration and Mapping
,”
Int. J. Rob. Res.
,
33
(
4
), pp.
519
533
.
40.
Gerkey
,
B.
,
Vaughan
,
R. T.
, and
Howard
,
A.
,
2003
, “
The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems
,”
Proceedings of the 11th International Conference on Advanced Robotics
,
Coimbra, Portugal
,
June 30
, Vol.
1
,
Citeseer
, pp.
317
323
.
41.
Brabbs
,
J.
,
Haynes
,
B.
, and
Stanko
,
T.
,
2020
, “
Using a Gaming Engine for Autonomous Vehicle Modeling and Simulation
,”
Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium
,
Novi, MI
,
Aug. 11
, pp.
11
13
.
42.
Durst
,
P. J.
,
Goodin
,
C.
,
Cummins
,
C.
,
Gates
,
B.
,
Mckinley
,
B.
,
George
,
T.
,
Rohde
,
M. M.
,
Toschlog
,
M. A.
, and
Crawford
,
J.
,
2012
, “
A Real-Time, Interactive Simulation Environment for Unmanned Ground Vehicles: The Autonomous Navigation Virtual Environment Laboratory (anvel)
,”
2012 Fifth International Conference on Information and Computing Science (ICIC)
,
Liverpool, UK
,
July 24
,
IEEE
, pp.
7
10
.
43.
Barlow
,
M.
, and
Garratt
,
M.
,
2020
, “
Autonomous Recognition of Collective Behaviour in Robot Swarms
,”
AI 2020: Advances in Artificial Intelligence: 33rd Australasian Joint Conference
, Vol.
12576
, Canberra, ACT, Nov. 29–30,
Springer Nature
, p.
281
.
44.
Rooban
,
S.
,
Suraj
,
S. D.
,
Vali
,
S. B.
, and
Dhanush
,
N.
,
2021
, “
Coppeliasim: Adaptable Modular Robot and Its Different Locomotions Simulation Framework
,”
Mater. Today: Proc.
,
1
(
1
), pp.
1
7
.
45.
Dosovitskiy
,
A.
,
Ros
,
G.
,
Codevilla
,
F.
,
López
,
A.
, and
Koltun
,
V.
,
2017
, “
Carla: An Open Urban Driving Simulator
,”
Proceedings of Machine Learning Research
,
Mountain View, CA
,
Nov. 13
.
46.
2021, OpenScenario, https://www.asam.net/standards/detail/openscenario/, Accessed July 1, 2021.
47.
Lee
,
T.-K.
,
Wang
,
T.-W.
,
Wu
,
W.-X.
,
Kuo
,
Y.-C.
,
Huang
,
S.-H.
,
Wang
,
G.-S.
,
Lin
,
C.-Y.
,
Chen
,
J.-J.
, and
Tseng
,
Y.-C.
,
2019
, “
Building a V2x Simulation Framework for Future Autonomous Driving
,”
2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS)
,
Matsue, Japan
,
Sept. 18
,
IEEE
, pp.
1
6
.
48.
Goodin
,
C.
,
Prevost
,
Z.
, and
Lemasson
,
B.
,
2015
, “
Simulation of Biologically-Inspired Control Algorithms for Teams of Ground Vehicles
,”
Conference on Autonomous and Robotic Construction of Infrastructure
,
Ames, IA
,
June 2
, p.
105
.
49.
Goodin
,
C.
,
Carrillo
,
J. T.
,
McInnis
,
D. P.
,
Cummins
,
C. L.
,
Durst
,
P. J.
,
Gates
,
B. Q.
, and
Newell
,
B. S.
,
2017
, “
Unmanned Ground Vehicle Simulation With the Virtual Autonomous Navigation Environment
,”
2017 International Conference on Military Technologies (ICMT)
,
Brno, Czech Republic
,
May 31
,
IEEE
, pp.
160
165
.
50.
Kuru
,
K.
, and
Khan
,
W.
,
2020
, “
A Framework for the Synergistic Integration of Fully Autonomous Ground Vehicles With Smart City
,”
IEEE Access
,
9
, pp.
923
948
.
51.
Gorsich
,
D. J.
,
Jayakumar
,
P.
,
Cole
,
M. P.
,
Crean
,
C. M.
,
Jain
,
A.
, and
Ersal
,
T.
,
2018
, “
Evaluating Mobility Performance of Unmanned Ground Vehicles
,”
Tech. Rep., US Army Tardec Warren
.
52.
Novick
,
D. K.
,
2020
, “
Market Survey of Airborne Small Unmanned Aircraft System Sensors
,” February 2020, Tech. Rep., Sandia National Lab. (SNL-NM), Albuquerque, NM.
53.
Skoglund
,
M.
,
Petig
,
T.
,
Vedder
,
B.
,
Eriksson
,
H.
, and
Schiller
,
E. M.
,
2016
, “
Static and Dynamic Performance Evaluation of Low-Cost RTK GPS Receivers
,”
2016 IEEE Intelligent Vehicles Symposium (IV)
,
Gotenburg, Sweden
,
June 19
,
IEEE
, pp.
16
19
.
54.
Gropp
,
W.
,
Thakur
,
R.
, and
Lusk
,
E.
,
1999
,
Using MPI-2: Advanced Features of the Message Passing Interface
,
MIT Press
,
Cambridge, MA
.
55.
Rankin
,
A.
,
Huertas
,
A.
, and
Matthies
,
L.
,
2005
, “
Evaluation of Stereo Vision Obstacle Detection Algorithms for Off-Road Autonomous Navigation
,”
Tech. Rep., NASA-JPL
.
56.
Hart
,
P. E.
,
Nilsson
,
N. J.
, and
Raphael
,
B.
,
1968
, “
A Formal Basis for the Heuristic Determination of Minimum Cost Paths
,”
IEEE Trans. Syst. Sci. Cybernet.
,
4
(
2
), pp.
100
107
.
57.
Weide
,
H.
,
2018
, “
A-star
,” https://github.com/hjweide/pyastar2d
58.
Hu
,
X.
,
Chen
,
L.
,
Tang
,
B.
,
Cao
,
D.
, and
He
,
H.
,
2018
, “
Dynamic Path Planning for Autonomous Driving on Various Roads With Avoidance of Static and Moving Obstacles
,”
Mech. Syst. Signal Process.
,
100
, pp.
482
500
.
59.
Coulter
,
R. C.
,
1992
, “
Implementation of the Pure Pursuit Path Tracking Algorithm
,” Tech. Rep., Robotics INST, Carnegie-Mellon UNIV, Pittsburgh, PA.
60.
Matsumura
,
S.
,
Omatu
,
S.
, and
Higasa
,
H.
,
1993
, “
Speed Control of an Electric Vehicle System Using PID Type Neurocontroller
,”
Proceedings of 1993 International Conference on Neural Networks (IJCNN-93)
, Vol.
1
,
Nagoya, Japan
,
Oct. 25
,
IEEE
, pp.
661
664
.
61.
Goodin
,
C.
,
Carruth
,
D.
,
Doude
,
M.
, and
Hudson
,
C.
,
2019
, “
Predicting the Influence of Rain on Lidar in Adas
,”
Electronics
,
8
(
1
), p.
89
.
62.
Dabbiru
,
L.
,
Goodin
,
C.
,
Scherrer
,
N.
, and
Carruth
,
D.
,
2020
, “
Lidar Data Segmentation in Off-Road Environment Using Convolutional Neural Networks (CNN)
,”
SAE Int. J. Adv. Current Pract. Mobil.
,
2
(
2020-01-0696
), pp.
3288
3292
.
63.
Foroutan
,
M.
,
Tian
,
W.
, and
Goodin
,
C. T.
,
2020
, “
Assessing Impact of Understory Vegetation Density on Solid Obstacle Detection for Off-Road Autonomous Ground Vehicles
,”
ASME Lett. Dyn. Syst. Control
,
1
(
2
), p.
021008
.
64.
Wald
,
I.
,
Woop
,
S.
,
Benthin
,
C.
,
Johnson
,
G. S.
, and
Ernst
,
M.
,
2014
, “
Embree: A Kernel Framework for Efficient CPU Ray Tracing
,”
ACM Trans. Graph.
,
33
(
4
), p.
143
.
65.
Creighton
,
D. C.
,
McKinley
,
G. B.
,
Jones
,
R. A.
, and
Ahlvin
,
R. B.
,
2009
, “
Terrain Mechanics and Modeling Research Program: Enhanced Vehicle Dynamics Module
,”
Tech. Rep., Engineer Research and Development Center, Vicksburg, MS, Geotechnical and Structural Labaoratory
.
66.
Park
,
H. C.
, and
Baek
,
N.
,
2020
, “Design of Selfengine: A Lightweight Game Engine,”
Information Science and Applications
,
H.
Kim
,
K. J.
Kim
, and
S.
Park
, eds.,
Springer
,
New York
, pp.
223
227
.
67.
Davis
,
D. C.
,
1975
, “
A Radial-Spring Terrain-eEveloping Tire Model
,”
Veh. Syst. Dyn.
,
4
(
1
), pp.
55
69
.
68.
Mason
,
G. L.
,
Williams
,
J. M.
,
Vahedifard
,
F.
, and
Priddy
,
J. D.
,
2018
, “
A Unified Equation for Predicting Traction for Wheels on Sand Over a Range of Braked, Towed, and Powered Operations
,”
J. Terramech.
,
79
, pp.
33
40
.
69.
Williams
,
J. M.
,
Vahedifard
,
F.
,
Howard
,
I. L.
,
Borazjani
,
A.
,
Mason
,
G. L.
, and
Priddy
,
J. D.
,
2019
, “
Mobility Guidance for Tracked Vehicles on Fine-Grained Soil From Historical Full-Scale Test Data in Drove 2.0
,”
J. Terramech.
,
84
, pp.
1
12
.
70.
Moore
,
M.
,
Ray
,
P.
,
Hudson
,
C.
,
Goodin
,
C.
,
Doude
,
M.
,
Carruth
,
D.
,
Ewing
,
M.
, and
Towne
,
B.
,
2020
, “
Exploring the Requirements and Capabilities of Off-Road Simulation in MAVS and Gazebo
,”
Proc. Ground Veh. Syst. Eng. Technol.
,
1
(
1
), pp.
13
15
.
71.
Islam
,
F.
,
Nabi
,
M.
,
Farhad
,
M. M.
,
Peranich
,
P.
,
Ball
,
J. E.
, and
Goodin
,
C.
,
2021
, “
Evaluating Performance of Extended Kalman Filter Based Adaptive Cruise Control Using PID Controller
,”
Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2021
,
Orlando, FL
,
Apr. 12
, Vol.
11748
,
International Society for Optics and Photonics
, p.
1174807
.
72.
Moh
,
M.
, and
Nguyen
,
B.
,
2003
, “
Qos-Guaranteed One-to-Many and Many-to-Many Multicast Routing
,”
Comput. Commun.
,
26
(
7
), pp.
652
669
.
73.
Dagum
,
L.
, and
Menon
,
R.
,
1998
, “
Openmp: An Industry Standard API for Shared-Memory Programming
,”
IEEE Comput. Sci. Eng.
,
5
(
1
), pp.
46
55
.
74.
Weigeshoff
,
W. R.
,
1992
, “
An Automated Reconnaissance and Surveillance Planning Tool
,”
Tech. Rep.
,
Naval Postgraduate School
,
Monterey, CA
.
75.
Lee
,
M. G.
,
Park
,
Y. K.
,
Jung
,
K. K.
, and
Yoo
,
J. J.
,
2011
, “
Estimation of Fuel Consumption Using In-Vehicle Parameters
,”
Int. J. u- and e-Serv. Sci. Technol.
,
4
(
4
), pp.
37
46
.
76.
Dollar
,
P.
,
Wojek
,
C.
,
Schiele
,
B.
, and
Perona
,
P.
,
2011
, “
Pedestrian Detection: An Evaluation of the State of the Art
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
34
(
4
), pp.
743
761
.
77.
Ulrich
,
I.
, and
Borenstein
,
J.
,
1998
, “
Vfh+: Reliable Obstacle Avoidance for Fast Mobile Robots
,”
Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146)
,
Leuven, Belgium
,
May 16
, Vol.
2
,
IEEE
, pp.
1572
1577
.
78.
Kaparias
,
I.
,
Bell
,
M. G.
, and
Belzner
,
H.
,
2008
, “
A New Measure of Travel Time Reliability for In-Vehicle Navigation Systems
,”
J. Intell. Trans. Syst.
,
12
(
4
), pp.
202
211
.
79.
Mun
,
J.
,
2008
,
Advanced Analytical Models: Over 800 Models and 300 Applications From the Basel II Accord to Wall Street and Beyond
, Vol. 419,
John Wiley & Sons
,
Hoboken, NJ
.
80.
Zhou
,
X.-H.
,
Gao
,
S.
, and
Hui
,
S. L.
,
1997
, “
Methods for Comparing the Means of Two Independent Log-Normal Samples
,”
Biometrics
,
53
(
3
), pp.
1129
1135
.
81.
Olsson
,
U.
,
2005
, “
Confidence Intervals for the Mean of a Log-Normal Distribution
,”
J. Stat. Educ.
,
13
(
1
), pp.
161
166
.
82.
Larson
,
J.
, and
Trivedi
,
M.
,
2011
, “
Lidar Based Off-Road Negative Obstacle Detection and Analysis
,”
2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC)
,
Washington, DC
,
Oct. 5
,
IEEE
, pp.
192
197
.
83.
Nahin
,
P. J.
,
2013
, “18. Average Distances on a Square,”
Will You Be Alive 10 Years From Now
,
P. J.
Nahin
, ed.,
Princeton University Press
,
Princeton, NJ
, pp.
126
138
.