Google DeepMind sees HBF memory as key to AI inference

This week, Sandisk and SK hynix unveiled memory of the HBF standard, which combines large storage capacity with high interface bandwidth. The development was also backed at the industry consortium level by Tenstorrent and Google, whose representatives noted that HBF will prove itself in inference. Inference refers to the process of drawing conclusions from already trained AI models. HBF, in terms of layout, resembles HBM memory but, unlike it, allows long-term data storage. The ability to quickly retrieve data for inference computations, according to Google DeepMind researcher Xiaoyu Ma, will be very much in demand given current trends in AI systems. Consequently, demand for HBF from AI computing infrastructure will also grow. Additionally, it provides high energy efficiency, which is important for data center operators. However, experts are already concerned about the intensity of HBF usage in such systems, which could theoretically lead to rapid failure of the chips. They suggest organizing infrastructure so that HBF is used mainly for reading data, not writing. To compensate for the speed imbalance, HBF should be used in combination with HBM, which would be used for write operations.