We all know the scene. Two detectives on a cop show stand in a dimly lit room filled with monitors, reviewing surveillance images. A tech guy (yes, it’s almost always a guy) queues up image after image as the detectives look on, squinting at the screen in concentration. “There’s nothing here!” one detective insists. They’re about to give up, when the other detective (our hero) shouts, “Wait!”
我们都知道现场。 警察表演中的两名侦探站在昏暗的房间里，房间里装满监视器，查看监视图像。 当侦探望着镜头时，一个技术人员(是的，几乎总是一个人)在一个接一个的图像中排队，凝视着屏幕。 “这里什么都没有！” 一名侦探坚持。 当另一名侦探(我们的英雄)大喊：“等等！”时，他们将放弃。
Everyone stops. “Zoom in there!” the detective says. The tech guy obligingly zooms in on a grainy corner of the image. “Enhance that!” the detective intones. The tech guy taps some keys, mutters something about algorithms, and suddenly the image comes into focus, revealing some tiny, significant detail. The case is cracked wide open!
大家停下来 “放大那里！” T他的侦探说。 技术人员会努力放大图像的颗粒状角落。 “增强！” 侦探声调。 技术人员轻按一些键，喃喃自语一些算法，然后图像立即成为焦点，揭示了一些微小的重要细节。 案子开得裂了！
This scene is a crime drama cliché so pervasive that it has inspired its own meme video with nearly a million views.
Scenes like these drive real tech people bananas, because “zoom and enhance” has always seemed like an impossible fantasy. Until now. Thanks to two recent innovations, zoom and enhance is finally here. It has the potential to radically change police surveillance, often in concerning ways — or at least help you bring back your photos from the early ’00s.
诸如此类的场景推动了真正的技术人员的成长，因为“ 缩放和增强 ”一直看起来像是不可能的幻想。 到现在。 多亏了最近的两项创新，缩放和增强终于到了。 它有可能从根本上改变警察的监视方式，通常以令人关注的方式进行，或者至少可以帮助您恢复20年代初期的照片。
The first innovation behind real-life zoom and enhance comes from the world of photography. Until recently, photographers had two primary options for digital cameras: professional DSLRs like the Nikon D series, or cheap compact consumer cameras, like the kind you’d use for birthday or travel snapshots. DSLRs take great photos, but they’re bulky and conspicuous and can be hard to operate — not a great combo for surveillance work. Compact cameras rarely have the quality necessary for surveillance professionals.
That all began to change around 2015, with the rise of mirrorless cameras. These cameras have the tiny form factor of a compact camera, but thanks to advances in imaging chips driven in part by smartphones, they pack in the same high-quality image sensors usually found in a DSLR. Increasingly, they also borrow complex image processing software from the smartphone world, further enhancing their capabilities. And crucially, they allow for the use of professional lenses — easily the most important factor for taking high-quality photos.
For a few thousand dollars, a surveillance professional or police force can now purchase tiny, easy-to-use cameras that take better photos than the best professional cameras from just a few years ago.
The end result is a tiny camera that you can carry and use inconspicuously, while taking extremely detailed, high-resolution photos. The Q, a mirrorless camera from legendary German camera maker Leica, largely kicked off the trend. The latest Q model weighs just 1.4 pounds and takes 47-megapixel photos through an obscenely crisp lens that sees more detail than the human eye. With an ISO rating of 50,000 (15 times higher than that achieved by the fastest analog films), it can also essentially see in the dark.
Lower-priced competitors, like the Sony Alpha, have since emerged. For a few thousand dollars, a surveillance professional or police force can now purchase tiny, easy-to-use cameras that take better photos than the best professional cameras from just a few years ago. Zooming into photos taken on these cameras can sometimes feel like using zoom and enhance. The detail they capture — especially paired with modern software — is remarkable.
此后出现了诸如Sony Alpha之类的低价竞争对手。 监视专业人员或警察部队现在只需花费几千美元，就可以购买微型，易于使用的相机，这些相机比几年前最好的专业相机拍摄的照片更好。 放大在这些相机上拍摄的照片有时会感觉像是使用变焦和增强功能。 他们捕获的细节(尤其是与现代软件搭配使用)非常出色。
But combine mirrorless camera images with compressive sensing, and zoom and enhance is truly here. Compressive sensing allows you to massively enlarge an image without a major loss in quality. The tech has been around since the early 2000s, but it gained prominence in 2010 when researchers showed how it could be used to reconstruct an image of President Barack Obama using a tiny sample of randomly distributed pixels.
In 2017, Google showed how principles of compressive sensing could be combined with neural networks to reconstruct degraded or low-quality images in a process called A.I. super-resolution. The tech works by starting with sample images — often of faces or rooms — and deliberately messing them up by making them blurry, running them through a terrible JPEG compression system, and the like.
A neural network then looks at the degraded images, compares them to their high-quality counterparts, and learns how the two relate. Essentially, the network teaches itself all the ways that a digital image can degrade. Once it knows this, the process is reversed. The system is handed a low-quality or degraded image, and based on its training, it constructs a high-quality, undegraded version from scratch.
然后，神经网络查看降级的图像，将其与高质量的图像进行比较，并了解两者之间的关系。 本质上，网络会自学数字图像可能降级的所有方法。 一旦知道这一点，该过程就被逆转。 该系统将获得低质量或降级的图像，并基于其培训，从头开始构建高质量，未降级的版本。
Though Google has since largely exited the field, A.I. super-resolution has taken off. Services like Big JPG allow users to upload a low-quality photograph and have it instantly upscaled 400% or more, often with minimal loss of quality. Photoshop plugins have delivered similar tech to photographers, who use it to remove blurriness and sharpen images. My A.I.-driven photography company often uses the tech to upscale digital camera photos taken in the early 2000s, allowing even these low-quality early images to meet today’s standards for use in publications.
The tech, though, is also being used for surveillance. Quickly after its development, researchers began to show how the super-resolution could be used to upscale low-resolution surveillance photos or frames from surveillance videos. Others focused on using the tech for targeted applications, like license plate recognition. And many groups have focused on super-resolution for facial recognition images, going so far as to develop specialized algorithms for enhancing facial images.
Several vendors have integrated these algorithms into dedicated software products. Topaz Labs, in my experience, is the most advanced. Pair its Gigapixel AI product with the output of a modern mirrorless camera, and you’ve got zoom and enhance that rivals the imagined systems on shows like CSI.
多家供应商已将这些算法集成到专用软件产品中。 以我的经验， Topaz Labs是最先进的。 将其Gigapixel AI产品与现代无反光镜相机的输出配合使用，您将获得与CSI这类可与想象中的系统匹敌的变焦和增强功能。
Here, for example, is a photo of a Jamba Juice restaurant in Marin County, California, taken on my Leica Q mirrorless camera.
例如，这是使用我的Leica Q无反光镜相机拍摄的加利福尼亚州马林县Jamba Juice餐厅的照片。
I took this from across a street, with the palm-sized camera hanging around my neck. I then ran the photo through Topaz’s Gigapixel AI software, upscaling it 400% and using the company’s proprietary face reconstruction and sharpening algorithms.
我从一条街对面拿来的，手掌大小的相机挂在脖子上。 然后，我通过Topaz的Gigapixel AI软件运行该照片，将其放大400％，并使用该公司专有的面部重建和锐化算法。
Zooming in to full size on the enhanced image, you can see some incredible detail. Through the restaurant’s front window, you can clearly see a patron waiting in line and examining a menu.
You can even see that he’s wearing a blue surgical mask. Great job staying safe, unknown smoothie man! Flyers posted on the door are also visible, including some of the graphics on the flyer. You can see patrons inside placing their orders.
您甚至可以看到他戴着蓝色的外科口罩。 保持安全的好工作，不知名的冰沙人！ 张贴在门上的传单也可见，包括传单上的一些图形。 您会看到顾客在下订单。
Zooming and enhancing another part of the image, you can see the text on signs in the far background (“Jamba Curbside Pickup”) and how they’ve been attached to pillars using yellow tape. And in the far distance, you can see the mannequins in another nearby store and diners eating at outdoor tables.
缩放并增强图像的另一部分，您可以在远处的背景上看到招牌上的文字(“ Jamba路边拾音器”)以及如何使用黄色胶带将其连接到Struts上。 在远处，您可以看到附近的另一家商店里的人体模特和在户外餐桌上用餐的食客。
With more extreme zoom and a tweak to exposure, you can clearly make out the store’s signature Blendtec blenders on the counter inside.
Blender identification, of course, is not the most groundbreaking use of a new technology. But when you apply zoom and enhance in a surveillance context, things get scary fast.
Here, for example, is a photo I took of a Black Lives Matter protest in Times Square in 2016.
Applying zoom and enhance, you can clearly see the faces of police officers in the far back of the crowd. With facial reconstruction applied, these images would likely be good enough to find matches in a facial recognition database.
Combining this tech with facial recognition systems like Clearview AI would make it trivial to identify large numbers of people in a crowd of protesters. A plainclothes police officer or federal agent posing as a tourist could easily walk through a crowd of protesters while snapping photos on a tiny mirrorless camera. The photos could be run through a super-resolution system, enlarging them massively and enhancing the faces present.
将该技术与Clearview AI等面部识别系统结合使用，可以轻松地识别出一群抗议者中的大量人员。 伪装成游客的便衣警察或联邦特工可以轻松地穿过一群抗议者，同时用微型无反相机拍摄照片。 这些照片可以通过超分辨率系统运行，可以对其进行大规模放大并增强当前的面Kong。
Individual faces could then be pulled out of the image and run through a system like Clearview’s to identify every individual by name. Police forces and other agencies are reportedly already using A.I. to identify different actions (like breaking into a vehicle or loitering) and to search surveillance images for people based on their physical descriptions. It’s unclear if any are using super-resolution yet, but undoubtedly that will come. Face reconstruction tech will likely improve as well — many faces today still come out distorted when enhanced, but facial reconstruction errors will likely diminish with time.
然后可以将个人面部从图像中拉出，并通过像Clearview这样的系统运行，以通过名称识别每个个人。 据报道，警察部队和其他机构已经在使用AI识别不同的动作(例如闯入车辆或游荡)，并根据其身体描述搜索监视图像。 尚不清楚是否正在使用超分辨率，但是无疑会出现。 面部重建技术也可能会得到改善-如今，许多面部在增强后仍然会变形，但是面部重建错误可能会随着时间而减少。
We need to ensure that technologies like zoom and enhance are available to law enforcement when they’re truly needed. But we also need to make sure that they’re not abused.
As the tech improves, you might not even need a mirrorless camera or other high-quality cameras. Super-resolution may ultimately become good enough to perform zoom-and-enhance functions on the low-resolution output of a traditional surveillance camera, identifying every individual in a crowd using footage from traffic cams, surveillance cameras from a store or nearby home, or even a circling drone. It could also one day be applied to photos taken on a smartphone or even the low-resolution photos displayed on social media platforms like Instagram.
随着技术的进步，您甚至可能不需要无反光镜相机或其他高质量的相机。 超分辨率最终可能会变得足够好，可以在传统监控摄像头的低分辨率输出上执行缩放和增强功能，使用交通摄像头，商店或附近家庭的监控摄像头的镜头识别人群中的每个人，或者甚至是盘旋的无人机。 它也可能有一天应用于在智能手机上拍摄的照片，甚至是在Instagram等社交媒体平台上显示的低分辨率照片。
As with any new surveillance technology, ensuring responsible use of zoom and enhance is a matter of establishing the right laws and policies. The Fourth Amendment of the U.S. Constitution already provides protection against searches without a warrant. Courts have weighed issues of new tech in the past — for example, looking at whether surveillance with telephoto lenses violates the Fourth Amendment. They have generally ruled that widely available tech like zoom lenses can be used in many contexts, but specialized tech like radar that sees through walls cannot.
It’s not yet clear where zoom and enhance would fall on that spectrum. The technology might be viewed as just another version of the zoom lens on a traditional camera. But given its elements of artificial intelligence, courts might find that it’s too specialized of a technology to be mobilized without proper search warrants.
目前尚不清楚缩放和增强将在该频谱上落在何处。 该技术可能只是传统相机上变焦镜头的另一个版本。 但是考虑到人工智能的要素，法院可能会发现，它过于专业化，无法在没有适当搜查令的情况下进行动员。
For now, the tech is too new for these precedents to have been established. As citizens, the best thing we can do is to be aware of its existence. If you’re at a protest or another sensitive event, assume that you’re being surveilled and photographed. Even if you don’t see someone with a professional-looking camera, authorities could still be capturing your image at a high enough quality to look you up using facial recognition and identify you by name.
就目前而言，这项技术对于这些先例尚不成熟。 作为公民，我们能做的最好的事情就是意识到它的存在。 如果您在抗议或其他敏感事件中 ，请假设您正在接受监视和拍照。 即使您看不到带有专业外观的相机的人，当局也可能会以足够高的质量捕获图像，从而可以使用面部识别功能查找您并按名称识别您。
We can also proactively inform lawmakers about which new technologies we’re comfortable with and which ones we’re not. Popular anger over facial recognition technologies led to a proposed bill to ban the use of this tech in policing. We need to ensure that technologies like zoom and enhance are available to law enforcement when they’re truly needed. But we also need to make sure that they’re not abused.
我们还可以主动告知立法者哪些技术适合我们，哪些技术不适合。 对面部识别技术的普遍愤怒导致提议的一项法案禁止在警务中使用该技术。 我们需要确保缩放和增强等技术在真正需要时可供执法人员使用。 但是我们还需要确保它们未被滥用。
Much as science fiction did a good job of preparing us for space travel and computers, shows like CSI have done a good job of introducing us to the concept of zoom and enhance before it existed. But when you move beyond the imagined world of a good-guy cop fighting evil criminals, the real-world ethics of tech like zoom and enhance get blurry fast.