Silicon Photonics and CPO Target AI Data Bottlenecks

In SK hynix Newsroom’s third Tech Note, Professor Kim Young-hyun of Hanyang University ERICA’s Department of Next-Generation Semiconductor Convergence Engineering explains AI factory data bottlenecks and the roles of silicon photonics and co-packaged optics (CPO). AI semiconductor performance depends on more than the computing speed of individual GPUs: training data and model weights must arrive from memory on time, while networks must carry results between GPUs. As GPU computing speeds up, memory and network transfer capacity become more important.

Copper wiring used to connect server semiconductors works efficiently over short distances, such as between chips. But electrical signals weaken and interfere with one another as speed and distance increase. Repeaters and correction circuits that restore signals consume more power and add heat and cooling demands. Fiber optics can increase transmission capacity through wavelength-division multiplexing (sending separate data streams on different light wavelengths through one fiber). The proposed division of labor keeps copper inside chips and on short links, while optical connections extend across data-center racks and buildings.

Silicon photonics puts optical waveguides that carry light, modulators that encode information onto light, and photodetectors that convert arriving light into electrical signals onto small chips. Semiconductor microfabrication and mass-production methods can also be used for optical components, enabling multiple optical paths to be made and integrated near electronic circuits. Silicon, however, does not generate light efficiently, so communications lasers use compound semiconductors such as indium phosphide. An external laser source (ELS) keeps the laser outside the silicon photonics chip, away from hot electronic chips, and uses a replaceable module to simplify maintenance.

CPO places an optical engine, which converts electrical signals to light and back, next to a switch’s application-specific integrated circuit (ASIC) on the same package substrate. Conventional pluggable modules are easy to replace at the front of a device, but electrical signals must travel from the switch chip to the module. As switch capacity and port speeds rise, power consumption and signal-quality demands on this stretch increase. CPO shortens the electrical connection and assigns longer distances to fiber, creating room for greater bandwidth density in a limited switch area and lower power use for high-speed signal correction. It also requires co-design of optical components and electronic circuits, along with system-technology co-optimization (STCO). Challenges include laser supply, precise fiber connections, reliable operation amid heat around the ASIC, and managing defects and manufacturing yield that can affect the whole package.

Nvidia and Broadcom are accelerating commercialization by introducing CPO networking products. Meta disclosed an evaluation of a CPO switch system at the 2025 European Conference on Optical Communication (ECOC). It said optical-link power consumption in the test was about 65% lower than in the pluggable comparison, and that long-duration connection stability was confirmed under those test conditions. Two alternatives are linear pluggable optics (LPO), which retains a pluggable design while omitting the digital signal processor (DSP), and near-packaged optics (NPO), which places the optical engine close to the switch-chip package without putting it in the same package. The three approaches are options selected according to performance, power efficiency, cooling, and maintenance requirements, rather than successive stages.

HBM sits close to a GPU in the same package to shorten data travel to memory; CPO improves transmission efficiency across networks connecting multiple GPU systems. Even with sufficient data from HBM, a communications bottleneck between GPUs can make it difficult to improve overall performance. Memory pooling can ease shortages or idle capacity in system-specific memory by letting multiple processors share memory as a larger resource. CXL is a connectivity standard that supports memory expansion and pooling. Silicon photonics links could serve as access paths to pooled memory, but they cannot provide access to distant memory as quickly as HBM. The article says latency and power costs from data placement and movement need to be checked when HBM and optical memory pools are used together. It concludes that AI semiconductor competitiveness depends not only on GPU computing speed but also on the combination of memory, networks, optics, packaging, and system technologies.

In SK hynix Newsroom’s third Tech Note, Professor Kim Young-hyun of Hanyang University ERICA’s Department of Next-Generation Semiconductor Convergence Engineering explains AI factory data bottlenecks and the roles of silicon photonics and co-packaged optics (CPO). AI semiconductor performance depends on more than the computing speed of individual GPUs: training data and model weights must arrive from memory on time, while networks must carry results between GPUs. As GPU computing speeds up, memory and network transfer capacity become more important.

Copper wiring used to connect server semiconductors works efficiently over short distances, such as between chips. But electrical signals weaken and interfere with one another as speed and distance increase. Repeaters and correction circuits that restore signals consume more power and add heat and cooling demands. Fiber optics can increase transmission capacity through wavelength-division multiplexing (sending separate data streams on different light wavelengths through one fiber). The proposed division of labor keeps copper inside chips and on short links, while optical connections extend across data-center racks and buildings.

Silicon photonics puts optical waveguides that carry light, modulators that encode information onto light, and photodetectors that convert arriving light into electrical signals onto small chips. Semiconductor microfabrication and mass-production methods can also be used for optical components, enabling multiple optical paths to be made and integrated near electronic circuits. Silicon, however, does not generate light efficiently, so communications lasers use compound semiconductors such as indium phosphide. An external laser source (ELS) keeps the laser outside the silicon photonics chip, away from hot electronic chips, and uses a replaceable module to simplify maintenance.

CPO places an optical engine, which converts electrical signals to light and back, next to a switch’s application-specific integrated circuit (ASIC) on the same package substrate. Conventional pluggable modules are easy to replace at the front of a device, but electrical signals must travel from the switch chip to the module. As switch capacity and port speeds rise, power consumption and signal-quality demands on this stretch increase. CPO shortens the electrical connection and assigns longer distances to fiber, creating room for greater bandwidth density in a limited switch area and lower power use for high-speed signal correction. It also requires co-design of optical components and electronic circuits, along with system-technology co-optimization (STCO). Challenges include laser supply, precise fiber connections, reliable operation amid heat around the ASIC, and managing defects and manufacturing yield that can affect the whole package.

Nvidia and Broadcom are accelerating commercialization by introducing CPO networking products. Meta disclosed an evaluation of a CPO switch system at the 2025 European Conference on Optical Communication (ECOC). It said optical-link power consumption in the test was about 65% lower than in the pluggable comparison, and that long-duration connection stability was confirmed under those test conditions. Two alternatives are linear pluggable optics (LPO), which retains a pluggable design while omitting the digital signal processor (DSP), and near-packaged optics (NPO), which places the optical engine close to the switch-chip package without putting it in the same package. The three approaches are options selected according to performance, power efficiency, cooling, and maintenance requirements, rather than successive stages.

HBM sits close to a GPU in the same package to shorten data travel to memory; CPO improves transmission efficiency across networks connecting multiple GPU systems. Even with sufficient data from HBM, a communications bottleneck between GPUs can make it difficult to improve overall performance. Memory pooling can ease shortages or idle capacity in system-specific memory by letting multiple processors share memory as a larger resource. CXL is a connectivity standard that supports memory expansion and pooling. Silicon photonics links could serve as access paths to pooled memory, but they cannot provide access to distant memory as quickly as HBM. The article says latency and power costs from data placement and movement need to be checked when HBM and optical memory pools are used together. It concludes that AI semiconductor competitiveness depends not only on GPU computing speed but also on the combination of memory, networks, optics, packaging, and system technologies.