According to Qualcomm, its forthcoming technology can recognize particular objects like teeth, hair, facial hair, lips, cloth, and more. And this might make way for some intriguing opportunities.
The high-end smartphones expected to be released in 2023 will be powered by the new generation flagship processors from Qualcomm and MediaTek, which were both unveiled in the past few months. The OnePlus 11, Xiaomi 13 series, and Vivo X90 range are just a few examples of phones that have previously been released with these chipsets.
Although the processors add additional power, hardware-based ray tracing, and satellite connectivity, it appears that the year will be more about evolution than revolution in terms of traditional camera capabilities. High-end CPUs from Mediatek and Qualcomm don’t experience any significant modifications in terms of photo and video resolutions or frame rates.
However, camera support involves more than just resolution, and the two businesses do in fact deliver a number of significant under-the-hood imaging advancements, including professional video capture technology, improvements for 200MP sensors, and native RGBW camera compatibility. However, we’re also observing a trend toward integrating AI and imaging hardware, which in 2023 will at least allow for one fairly intriguing function.
More granular recognition
Real-time semantic segmentation is a feature Qualcomm is promoting for the Snapdragon 8 Gen 2. Semantic segmentation, for the uninitiated, is the capacity to distinguish particular objects and subjects within a frame. The ability of the camera software to recognize particular settings or persons and then perform image processing in accordance with those findings makes it a fundamental technology at the foundation of many camera modes.
Many smartphone brands employ semantic segmentation for single-camera portrait modes, while other brands use it for AI scene identification (sunsets, landscapes, flowers, food) (sunsets, landscapes, flowers, food). Even some companies, like Xiaomi and Google, have advertised the capacity to totally change the sky in a photograph, replacing a grey sky with an entirely blue one.
Qualcomm, though, is digging much further. For starters, the company affirms that its most recent version of the solution is quick enough to function for movies and in real-time. Additionally, it claims that their solution can recognize particular items like teeth, hair, facial hair, lips, cloth, and more. And this might make way for some intriguing opportunities.
Theoretically, Qualcomm’s improved semantic segmentation approach might lead to alternatives for cosmetic enhancements like teeth whitening.
However, manufacturers would have to tread carefully in order to avoid both propagating unrealistic beauty standards and offering people-wanted beautifying options. After all, over the years, we’ve seen a lot of dubious effects and filters, such face-thinning, nose-shaping, skin-lightening, and eye-widening.
However, more sophisticated semantic image segmentation is not just for aesthetic purposes. According to Qualcomm’s video, the technology may also make it possible for clothing to be processed more effectively, providing your jersey or jacket with additional sharpening without impacting the rest of your body. Even the capacity to clear glare from a pair of glasses is demonstrated in the video.
Real-time semantic segmentation is also programmable, according to Qualcomm. Therefore, businesses can use several neural networks if they have other ideas for the technology.
But will this technology be used in consumer smartphones?
Real-time semantic segmentation is great for chipmakers to support, but the real question is whether smartphones will ever arrive with this technology. After all, when it comes to incorporating a chipmaker’s camera functionality, smartphone manufacturers have a mixed track record (e.g. unlimited 960fps slow-motion, 120fps 12MP burst mode).
Heape fortunately confirmed that all smartphone manufacturers offered this capability “straight out of the box.” So there are no licensing payments, and the OEM has no further obligations, he said.
The merging of AI and imaging hardware
Qualcomm’s Hexagon Direct Link function makes it feasible to increase semantic segmentation. This speaks to Qualcomm’s successful establishment of a connection between the AI silicon and the ISP in charge of photo processing. With the Dimensity 9200 chipset, Mediatek is taking a same approach, claiming that it has combined AI and ISP technology for more effective 8K/30fps and 4K/60fps with electronic stabilization. While using AI hardware that is closely related to the imaging process, Google’s semi-custom Tensor chips found within Pixel phones also leverage this technology.
Mobile photography will get a variety of advantages from tightly integrated AI and imaging technology, including speed enhancements and brand-new functionality.
