It was a year ago (summer 2019) when I was deciding what topic to choose for my bachelor thesis. At the time I was finishing an internship at the European Space Agency where I worked on mineralogical machine learning research¹. However, I also knew that at my university there is a lot of research with self-driving cars in collaboration with Toyota. While mineralogical research that could be possibly used in future planetary missions is very important, self-driving cars are not only exciting, but also tangible and presentable. So I contacted my former research supervisor from my university whether he had any research ideas in this area. Among many proposed, I was most excited by Autonomous Car Chasing.

一年前(2019年夏季)，我正在决定为我的学士论文选择哪个主题。 当时，我正在欧洲航天局完成实习，在那里我从事矿物学机器学习研究 ¹。 但是，我也知道，在我的大学里，有很多与丰田合作的无人驾驶汽车研究。 虽然这可能在未来行星任务可能被用于矿物学研究是非常重要的，自动驾驶汽车不仅令人兴奋，但也是有形的和像样的 。 因此，我联系了我大学的前研究主管，他是否对此领域有任何研究想法。 在提出的众多建议中，自动驾驶汽车追逐令我最为兴奋。

为什么要追车？ (Why Car Chasing?)

When you look at autonomous cars in real-life, you will see that they are most often used in situations when the driving is too boring for a driver. They are used on a highway in a traffic jam or in a convoy driving at a near-constant speed. However, regular traffic forces the drivers to react to many unexpected situations and perform dynamic maneuvers. The proposed car chasing scenario, in which an autonomous car chases a non-cooperative vehicle that actively tries to drive away, is so extreme that an autonomous vehicle with car chasing capabilities can surely handle regular traffic.

在现实生活中观察自动驾驶汽车时，您会发现它们最常用于驾驶员无聊的情况。 它们用在高速公路上的交通拥堵中或以接近恒定的速度行驶的车队中。 但是，常规交通迫使驾驶员对许多意外情况做出React并进行动态操纵。 拟议的汽车追逐场景非常极端，以至于无人驾驶汽车追逐主动试图驶离的非合作车辆，因此具有汽车追逐能力的自动驾驶汽车肯定可以应付正常的交通。

新的具有挑战性的数据集 (New challenging dataset)

The first task was to create an autonomous car chasing system and test it in CARLA. To perform the experiments we first collected a new challenging publicly available CARLA Car Chasing Dataset collected by manually driving the chased car. We created a baseline algorithm, which detected the pursued car and then basically drove straight after it. This approach had a huge flaw. It wasn’t considering the surrounding environment while driving. It was only focused on the pursued car. Therefore, this system crashed a lot. We realized our system needed an understanding of the surrounding environment. We considered adding semantic segmentation neural network, but we already had a neural network detector. We needed to analyze as many frames per second as possible and also had a very limited computational power (embedded system on a RC car). We needed a fast and effective solution.

第一个任务是创建一个自动驾驶汽车追踪系统，并在CARLA中对其进行测试。 为了进行实验，我们首先收集了一个新的具有挑战性的公开可用的CARLA汽车追踪数据集 ，该数据集是通过手动驾驶被追踪的汽车而收集的。 我们创建了一个基准算法，该算法可以检测出所追逐的汽车，然后基本上在行驶后一直行驶。 这种方法有一个巨大的缺陷。 开车时没有考虑周围的环境。 它只专注于追逐的汽车。 因此，该系统崩溃很多。 我们意识到我们的系统需要了解周围的环境。 我们考虑过添加语义分割神经网络，但是我们已经有一个神经网络检测器。 我们需要每秒分析尽可能多的帧，并且计算能力也非常有限(RC汽车上的嵌入式系统)。 我们需要一种快速有效的解决方案。

双任务神经网络 (Dual-task neural network)

During inference, an image is passed through the network just once. The network provides an object detection as well as the semantic segmentation outputs. While the training is slightly slower than training a single-task neural network, the extra cost during inference of the proposed architecture is negligible. The segmentation output provides a semantic map of the input image consisting of 10x10 cells of two classes: a drivable surface and a background, see the image below.

在推理期间，图像仅通过网络传递一次。 网络提供对象检测以及语义分段输出。 虽然训练比训练单任务神经网络要慢一些，但是在推断所提出的体系结构期间的额外费用可以忽略不计。 分割输出提供输入图像的语义图，该输入图像由两类10x10的单元组成：可驱动的表面和背景，请参见下图。

The neural network shares the same backbone for both tasks — a 53 layer feature extractor called Darknet-53⁴. Attached to the feature extractor are two sets of layers — one that gives the output for the object detection and the other that gives an output for the image segmentation. The architecture of the neural network is depicted in the image below. The network is trained by alternating optimization — in every second batch, the network is optimized only for detection, while the segmentation is optimized in the remaining batches. The neural network uses different loss functions depending on the batch².

神经网络在两个任务上共享相同的主干-一个称为Darknet-53的53层特征提取器。 连接到特征提取器的是两组图层-一组为对象检测提供输出，另一组为图像分割提供输出。 下图描述了神经网络的体系结构。 通过交替优化来训练网络-在每第二批中，仅针对检测对网络进行优化，而在其余批次中对分段进行优化。 根据批次²，神经网络使用不同的损失函数。

实验结果 (Experiment results)

First, we tested the system using a difficult subset of the CARLA Car Chasing Dataset. We observed, that the full algorithm performed significantly better than other versions of the system that did not use coarse semantic segmentation. It achieved almost 10 percentage points higher drive completion on average than the next best-evaluated version. We also showed that the dual-task neural network system is more resistant to detector miss rate (when the detector failed to detect the pursued car).

首先，我们使用CARLA Car Chasing数据集的困难子集测试了系统。 我们观察到，完整算法的性能明显优于不使用粗略语义分割的其他版本的系统。 与下一个最佳评价版本相比，它的驱动器平均完成率提高了近10个百分点。 我们还表明，双任务神经网络系统对检测器未命中率的抵抗力更高(当检测器无法检测到所追捕的汽车时)。

Then, we performed several live tests under different weather and lighting conditions. The system was tested on an empty roundabout as well as in a residential area as depicted in the image below. The autonomous system followed the other car smoothly without jerky movements. For the most part, it was able to successfully chase the other vehicle. It was maintaining the desired distance when the pursued car was driving in a straight line. If the chased car stopped, so did the autonomous system. A limitation of the system comes from its current reactive nature, which in certain rides affected the ability to make a U-turn on a narrow road.

然后，我们在不同的天气和光照条件下进行了几次现场测试。 如下图所示，该系统在空的回旋处以及住宅区进行了测试。 自主系统平稳地跟随另一辆汽车，没有晃动。 在大多数情况下，它能够成功追逐另一辆车。 当被追赶的汽车以直线行驶时，它保持了所需的距离。 如果被追赶的汽车停了，那么自动驾驶系统也会停下来。 该系统的局限性在于它目前的React特性，在某些行驶过程中，它会影响在狭窄道路上掉头的能力。

会议 (Conference)

The results of the novel proposed approach were clear and before the bachelor thesis was even finished, we knew there was a potential for publication. After finishing polishing the thesis, we started its transformation into a research article. A multiple-page section in the thesis easily turned into a couple of lines in the publication. Unlike a thesis that mainly serves as proof of your understanding of the topic, we turned it into something that could be read by other people. We decided to publish to a workshop associated with the third most prestigious conference on computer vision called European Conference on Computer Vision (ECCV). The main advantage of a workshop is its short processing time. In two weeks, we knew our paper was accepted.

提出的新方法的结果很明确，甚至在学士学位论文尚未完成之前，我们就知道有发表的潜力。 完成论文的完善后，我们开始将其转化为研究论文。 论文中多页的部分很容易变成出版物中的两行。 与主要用来证明您对该主题的理解的论文不同，我们将其转变为其他人可以阅读的东西。 我们决定将与第三届最负盛名的计算机视觉会议 ( 欧洲计算机视觉会议 (ECCV))相关的研讨会发布。 车间的主要优点是加工时间短。 两周后，我们知道我们的论文被接受了 。

讨论区 (Discussion)

This paper isn’t the first (nor second) paper I have published. I am also not flying to the other side of the world to present it like with my first publication (FG 2018 in China) because the conference is only online/virtual due to COVID-19. That said, I am very proud of the paper. The project is very exciting and hopefully, other researchers (or students at FIT CTU) will attempt to improve on the method.

本文不是我发表的第一篇(也是第二篇)。 我也没有像我的第一本出版物(FG 2018在中国)那样飞往世界的另一端，因为由于COVID-19，这次会议只是在线/虚拟的。 就是说，我为这份文件感到骄傲。 该项目非常令人兴奋，希望其他研究人员(或FIT CTU的学生)将尝试改进该方法。

I believe that my motivation for the topic was the main reason for the success of the thesis. The thesis topic was exciting and also purposeful and I wish you to find a topic that you can get excited about. I hope there can be a collaboration between Center for Machine Perception (CMP) at FEL CTU and FIT CTU. If you are interested in autonomous driving or computer vision, CMP is a place to be. I can also definitely recommend my supervisor Jan Cech, who discussed the thesis progress with me on a regular weekly basis.

我相信我对该主题的动机是论文取得成功的主要原因。 论文主题既令人兴奋，又有针对性，我希望您能找到一个令您兴奋的主题。 我希望FEL CTU的机器感知中心(CMP)与FIT CTU之间可以进行合作。 如果您对自动驾驶或计算机视觉感兴趣，可以选择CMP。 我当然也可以推荐我的主管Jan Cech，他每周定期与我讨论论文的进展情况。

结论 (Conclusion)

We have developed a system, capable of autonomously chasing another vehicle, using the novel dual-task network that concurrently detects objects and predicts coarse semantic segmentation. The proposed system was extensively tested in CARLA simulator using a new challenging publicly available (at our GitHub) chasing dataset and on a real sub-scale vehicle platform (video available on youtube).

我们已经开发了一种系统，该系统能够使用新颖的双任务网络同时自动检测对象并预测粗略的语义分割，从而能够自动追逐另一辆车。 拟议的系统已在CARLA模拟器中使用新的具有挑战性的公开可用( 在我们的GitHub上 )的追踪数据集和真实的小规模车辆平台( 在youtube上提供了视频 )进行了广泛测试。

Despite the simplicity of the proposed system, it shows robust chasing capabilities by using only information from a single RGB camera. One of the system limitations is its reactive nature. We believe that the system could improve by using a more sophisticated trajectory planning algorithm that would include predictive modeling of the chased car.

尽管建议的系统很简单，但仅使用来自单个RGB摄像机的信息，它仍显示出强大的跟踪能力。 系统限制之一是其React性。 我们认为，可以通过使用更复杂的轨迹规划算法(包括对被追赶的汽车进行预测建模)来改善系统。

We hope to inspire other researchers or students looking for a thesis topic to try and improve our methods. We believe that testing an autonomous driving system under extreme conditions is an important step to creating public trust in self-driving cars. Ultimately, extreme challenges like the DARPA challenge, or our autonomous car chasing is what will drive innovation forward.

我们希望启发其他正在寻找论文主题的研究人员或学生尝试并改进我们的方法。 我们认为，在极端条件下测试自动驾驶系统是建立公众对自动驾驶汽车的信任的重要步骤。 最终，诸如DARPA挑战之类的极端挑战或我们的自动驾驶汽车追逐将推动创新向前发展。