Fujitsu Laboratories Ltd. announced the development of high-speed in-memory data deduplication technology for all flash devices which enables the production of storage systems with up to twice the response speed when writing data compared to previous methods. These flash arrays are large scale and have features like high speed storage systems and use multiple flash devices such as solid state drivers.
Recently all-flash arrays have incorporated deduplication technology that consolidates duplicate data into one to write to a flash device, in order to utilize the limited capacity of flash devices. However, as the system connects to multiple flash devices through a network in order to search for duplicate data each time it writes data, and storage devices grow in capacity and increase in speed, a problem of lowered response speed during write operations arises.
Now Fujitsu Laboratories has developed a new method that can accelerate response speeds by executing deduplication after writing data. In addition, as data may be written to memory twice in some cases when processing is continued with the new method, thereby increasing communications volume and lowering overall processing performance.
Fujitsu Laboratories has thus developed this technology to automatically switch between the new method and the previous method, as operational conditions require. This means that response speeds can be increased by up to two times, improving the response of virtual desktop services and reducing database processing times.
In recent years, with the advance of virtualization and the use of big data in business, performance requirements for storage systems have been increasing. All-flash arrays, which can increase performance by using high-speed flash devices and connecting multiple devices with a high-speed network, have therefore become widely used.
Flash devices are more expensive than traditional hard drives, and can only be written to a limited number of times over their lifetime, so all-flash arrays incorporate deduplication functionality that consolidates data to write only once to the flash device, which can increase the actual volume and reduce the number of unnecessary write operations.
With this technology, by automatically choosing the optimal method in response to the continually changing status of the system, it is possible to reduce system response time as a whole.
Fujitsu Laboratories will continue development of technologies to further accelerate all-flash arrays going forward, aiming to incorporate them into Fujitsu Limited's storage products from fiscal 2017 or later.