Issue 46 • November 2022

datacenterdynamics.com

The Last
Data
Center
Long-term data storage
enters a new epoch

Cable ships head for DE-CIX’s CEO A second look Targeting

rough seas Ivo Ivanov peers into Why big tech is taking cybercrime
Boat shortages the future on time Raiding data centers
 CUSTOMIZATION
IS OUR STANDARD

CONFIGURABLE
RACKS
MILLIONS OF CONFIGURATIONS AVAILABLE

AISLE
CONTAINMENT
NEW DATA CENTER BUILDS OR RETROFITS

COLOCATION
CAGES
SECURE AND FLEXIBLE LAYOUTS

www.amcoenclosures.com/data 847-391-8100
an IMS Engineered Products Brand
MADE IN THE USA
 ISSN 2058-4946

Contents
November 2022
6 News
Loudoun’s limits, bankruptcies, fires,
saboteurs, and layoffs. Plus the climate
crisis claims a data center

14 When everything else is gone

Exploring the frontiers of long-term
data storage to find out what remains
of the human race

24 The CEO interview

“We see a huge wave of new market
participants on platforms like DE-CIX,
who are not interested in peering
directly,” DE-CIX’s CEO says.

14 24 28 L
 ight matters
How photonic computing could
revolutionize AI and interconnect
technology

31 A second look
The dangers of a negative leap second
mean that its time to rethink time

35 Cable ship crisis

Everyone wants submarine cables.
Nobody wants to buy the ships that
lay them

The flamingo universe

42 
Simulating the cosmos requires a lot
of RAM.

46 A necessary shutdown

35 28
Why we’re killing off 2G and 3G, and
what it means for 5G

48 R
 aiding data centers
When law enforcement comes for
servers, what happens?

51 A
 mped up
Ampere talks Arm data centers, and
why it’s building its own cores

54 Telco consolidation
Behind Vodafone’s merger with
Three UK

58 Reusing waste compute

How digital boilers could heat homes,
and do compute on the side

48 54 62 Op-ed: Brace for impact

The economy is turning, and that
means tough times ahead

Issue 46 • November 2022 | 3

 GET THE POWER
OF THE PLUS.
  ™   

    
 



   

 

     ­

€‚ƒ„  †   

‡ˆ  ‰

Š 

        


>>CONTENTS

From the Editor Meet the team

10,000
Editor-in-Chief
Sebastian Moss

When everything
@SebMoss
Executive Editor
Peter Judge
@Judgecorp

else is gone News Editor

Dan Swinhoe
@DanSwinhoe

W
Telecoms Editor
hat will remain of money becomes less attainable (p62). Paul Lipscombe
the human race? Further down the line, we Reporter
We don't will start to suffer from years of Georgia Butler
know how long underinvestment in the cable ship Partner Content Editor
our species will industry, just as submarine cables start Claire Fletcher
survive, and even to boom. Head of Partner Content
if we can hold on for thousands more There's no incentive for operators Graeme Burton
@graemeburton
years it is not clear that the knowledge to build more, and that could mean a
of today will carry on with us. crisis for those that depend on them SEA Correspondent
Paul Mah
On the cover, we talk to those (p35). The number of @PaulMah
building records of our present to years Microsoft Brazil Correspondent
bring hope to our future. From a mine Darkness believes that Tatiane Aquim
at the end of the Earth, to Microsoft's One sector that relies on the growth of Project Silica @DCDFocuspt
cutting edge research labs, to the edge the Internet is that of cybercrime.
will be able to Designer
of space, we travel in search of the next But nefarious platforms, spammers, Eleni Zevgaridou
hold data for
stage of data storage. and criminal enterprises still need Head of Sales
somewhere to call a home. Erica Baeta
The world of cybercrime Conference
Data storage is nearing enforcement is not as glamorous as Director, Global
its limits, unless Hollywood might pretend, but it still Rebecca Davison
means the occasional data center raid. Conference
we embrace new Now, however, the denizens of the Director, NAM
Kisandka Moses
technologies dark web are embracing the cloud,
causing new challenges for law Channel Manager
enforcement (p48). Alex Dickins
Over in a second A different form of darkness Channel Manager
Emma Brooks
While that feature focuses on immense can be found in Durham. We travel
timescales, elsewhere we delve into to the North England city to see a Channel Manager
Gabriella Gillett-Perez
the impact of a single second. A fight supercomputer built to study dark
over time itself risks causing serious matter at immense scales. Chief Marketing Officer
Dan Loosemore
outages, as we face the first ever You can find the answer to life, the
negative leap second (p31). universe, and everything on page 42. Head Office
DatacenterDynamics
Connection issues The telco challenge 22 York Buildings,
John Adam Street,
Our vulnerability to perturbations in Like supermassive black holes, telecoms
London, WC2N 6JU
collective time are the result of our companies must consume all around
interconnected digital world. them.
How that came to be, and why it's Our new telco editor, Paul
important to build an open network, Lipscombe, analyzes the next big © 2022 Data Centre Dynamics Limited All
rights reserved. No part of this publication
are the focus of our interview with the merger in the telco sector - Vodafone may be reproduced or transmitted, in
any form or by any means, electronic,
CEO of DE-CIX (p24). and Three UK - and looks at what it mechanical, photocopying, recording
But we may not be able to keep means for their competitors (p54). Sebastian Moss or otherwise, or be stored in any
retrieval system of any nature, without
building out as easily as we have in the Elsewhere, he charts the end of 2G Editor-in-Chief prior written permission of Data Centre
Dynamics Limited. Applications for
written permission should be directed
past, as the economy starts to turn and and 3G services (p46). to the editorial team at editorial@
datacenterdynamics.com. Any views or
opinions expressed do not necessarily
represent the views or opinions of Data
Centre Dynamics Limited or its affiliates.
Disclaimer of liability: Whilst every effort
Dive even deeper has been made to ensure the quality and
accuracy of the information contained in
Follow the story and find out more about DCD products that can further expand your knowledge. this publication at the time of going to
press, Data Centre Dynamics Limited and
its affiliates assume no responsibility as
Each product is represented with a different icon and color, shown below. to the accuracy or completeness of and,
to the extent permitted by law, shall not
be liable for any errors or omissions or
any loss, damage or expense incurred by
reliance on information or any statement
contained in this publication. Advertisers
are solely responsible for the content
of the advertising material which they
submit to us and for ensuring that the
material complies with applicable laws.
Data Centre Dynamics Limited and its
affiliates are not responsible for any error,
omission or material. Inclusion of any
advertisement is not intended to endorse
any views expressed, nor products or
Events Intelligence Debates Training Awards CEEDA services offered, nor the organisations
sponsoring the advertisement.

Issue 46 • November 2022 | 5

 Whitespace
>>CONTENTS

News
The biggest data center news stories of
NEWS IN BRIEF

Meta ups server room tem-

peratures to 90 degrees F
The company also reduced humidity
to 13 percent. But as many of its large
the last three months facilities are in drought-stricken areas,
some are concerned about local impacts
even with wider restoration efforts.

Vertiv CEO Rob Johnson to

retire on health grounds
Giordano Albertazzi, currently President,
Americas, has been appointed
immediately as chief operating officer.
On January 1, 2023, Albertazzi will
succeed Johnson as CEO.

Microsoft installs Ambri

high-temperature ‘liquid
metal’ batteries as backup
The Ambri batteries use electrodes
made of calcium alloy and antimony
electrodes, with a molten salt
electrolyte. They are also used by
TerraScale data centers.

Stack deploys beehives at

data center campus in Milan
Loudoun County puts limits on data “Honeybees are responsible for 80

center growth percent of the world’s pollination,

but their population is threatened
Loudoun County, Northern Virginia, home boards, without the backing of residents and drastically declining,” Stack said.
of the world’s largest concentration of data - or even, apparently, without reference to “To support their preservation, we’ve
centers, has adopted new rules which could whether the utility, Dominion Power, could adopted three hives that will host
limit future development there. even supply them with electricity. nearly 200,000 bees.”
In late September, the Board of To fix this problem, Loudoun is
Supervisors of the County approved reclassifying neighborhoods, according to
proposals from the county planners, which the “place types” used by zoning rules. EcoDataCenter to reuse
will limit data center projects in some Data centers will no longer be approved heat in fish farms and
neighborhoods, particularly along Route 7. by right in “suburban mixed-use” locations,
greenhouses
The new rules will also require data centers which make up 887 acres of vacant land in
EcoDataCenter and Wa3rm will focus
to adopt higher-quality building designs and the County. They will also lose automatic
on large-scale cultivation in fish farms
tougher environmental rules depending on approval in areas including “urban
and greenhouses in the projects,
their proximity to housing. transit center,” “urban mixed-use,” “urban
though further details haven’t been
The new zoning rules, based on proposals employment,” and “suburban neighborhood.”
shared. There are several data centers
set out by the planners in July, were adopted They will also be barred in “urban transit
warming fish farms, including Green
by the Board of Supervisors on September 20. center” areas, where tall buildings cluster
Mountain and the White Data Center.
The adopted rules will exclude data around Metrorail stations - and that place
centers from certain areas, in particular type has been enlarged by also including
keeping them Route 7, where there is no “urban employment” areas. Switch begins work on
more power infrastructure to support them. Data centers continue to be allowed by- expanding pyramid campus
Route 28 will remain open to new facilities. right in areas designated “suburban industrial/ in Grand Rapids, Michigan
The problem on Route 7 arose because mineral extraction”, “transition light industrial,” Gaines Township officials approved a
data centers were approved automatically in and “transition industrial/mineral extraction.” site plan amendment in July 2021 for
many locations along the highway, so many Data centers that are permitted will a 312,000-square-foot (29,000 sqm)
were approved that the number exceeded face stricter environmental controls, using expansion at Switch’s existing data
the capacity of Loudoun’s power distribution “high-quality building design” to make their center, located at the former Steelcase
networks. This happened because data exteriors less ugly, and limiting the noise Inc. “pyramid” building in Gaines
centers were approved “by right” in certain made by air-conditioning systems used to Township. Construction appears to
types of neighborhood. They were fast- cool the servers have begun in October this year.
tracked without any hearings with local bit.ly/LoudounBreakingPoint

6 | DCD Magazine • datacenterdynamics.com

>>CONTENTS DCD Magazine #46

Founded in 2017, Compute North

operates four US data centers; two in Texas
in McCamey (280MW) and Big Spring, and
one each in Kearney, Nebraska (100MW),
and North Sioux City, South Dakota (&MW).
A third 300MW facility was in development
in Granbury, Texas. The company’s assets
are worth between $100 million and $500
million, according to its Chapter 11 petition.
“The Company has initiated voluntary
Chapter 11 proceedings to provide the
company with the opportunity to stabilize its
business and implement a comprehensive
restructuring process that will enable
us to continue servicing our customers
and partners and make the necessary
investments to achieve our strategic

Compute North files for Chapter 11 objectives,” a spokesperson said.

Both Compass Mining and Marathon

bankruptcy, with $500m owed Digital are customers of Compute North,

and released statements saying the filing
Cryptomining data center firm Compute Bloomberg reports Compute North faced shouldn’t affect operations.
North has filed for bankruptcy. delays in energizing mining machines for However, Marathon has $31.3 million
Minnesota-based Compute North its client Marathon Digital Holdings Inc. in invested in Compute North, and has paid
Holdings Inc., which provides data center Texas due to local regulations in the state. the company about $50 million in security
hosting services for cryptocurrency miners Compute North’s 280MW mining facility deposits and prepayment.
and blockchain companies, filed for Chapter in Texas was planned to be operational Separately, rival cryptomining firm Core
11 bankruptcy in Texas this September. in April, but was held up by the approvals Scientific said that it was facing bankruptcy
The company said it owed as much as process. By the time it could launch, this October, warning that it would not pay
$500 million to at least 200 creditors. The Bitcoin prices had plummeted, and funding its debts for the month, or in November. By
firm’s CEO Dave Perrill has also stepped opportunities had dried up. the end of the year, it could run out of cash.
down but will remain on the board. In February, Compute North raised $385 “In the event of a bankruptcy proceeding
Data center operators worldwide are million, consisting of an $85 million Series or insolvency, or restructuring, holders of the
seeing energy prices rise, while cryptomining C equity round and $300 million in debt company’s common stock could suffer a total
firms are facing a double-whammy of low financing. However, one of its main backers, loss of their investment,” the company said.
Bitcoin prices and Ether’s recent move away Generate Capital, withdrew funding at the “Substantial doubt exists about the company’s
ability to continue as a going concern.
from ‘proof of work’ mining, reducing the end of the year, sparking the bankruptcy
bit.ly/CryptosGambleFails
need for mining hardware. process.

Dutch colo firm Datacenter Almere

files for bankruptcy
Dutch data center firm Datacenter Almere has filed for bankruptcy.
The Flevoland company this month was declared bankrupt by a court in
Midden-Nederland.
Led by Andrew van der Haar, the company offered colocation services
and operated a single data center on 153 Randstad 22 in Almere, outside
Amsterdam. Subsidiaries ICT Campus Almere and IaaS provider NL
Datastore are also subject to the bankruptcy decision.
The court has appointed a trustee, Mr. KCS Meekes, associated with
De Advocaten van Van Riet BV, who will take care of the settlement of the
bankruptcy. DCD has reached out to van der Haar for comment.
The facility was originally built in 2001 for Sara (now teaching and
research cooperative Surf.nl) before being taken over by Vancis in 2008.
Interxion/Digital Realty acquired the facility in 2018 and operated the site as
Almere ALM1. Datacenter Almere relaunched the facility in 2019.
The company reportedly suffered a major outage in September after a
break in a high-voltage cable.
A number of companies, including NorthC and Keppel, own and operate
data centers in Almere
bit.ly/DataCenterBankruptcy

Issue 46 • November 2022 | 7

 Whitespace >>CONTENTS

government, telecom operators, and Enedis

have discussed the issue over the summer.
“Maybe we’ll improve our knowledge on
the matter by this winter, but it’s not easy
to isolate a mobile antenna (from the rest of
the network),” said a French finance ministry
official with knowledge of the discussions.
Swedish, German, and Italian telcos
have also raised concerns, while Nokia and
Ericsson are working with mobile network
operators to mitigate the impact of potential
power shortages.
UK telecoms company BT recently told
the FT that it was not currently seeking
more backup power for the winter, but was
assessing which of its non-critical hardware
Europe could face a winter of mobile could be switched off.
In the data center industry, the likes of
network blackouts Equinix and Digital Realty have increased
their diesel reserves in preparation for
Mobile networks across Europe could start increases the prospect of mobile phone potential grid and fuel supply issues.
going down this winter, as operators warn outages. Equinix usually fills its tanks to 60 percent
that the energy crisis may lead to regular This has led some European countries capacity, but is now raising that to 90 percent
power cuts and energy rationing. to try and ensure communications can across many of its sites. Digital Realty said
This has led to fear within the telecoms resume as normal even if power cuts end up that it had established more priority delivery
industry, notes Reuters, which reports exhausting backup batteries. agreements with its diesel suppliers. One of
that industry officials are concerned that Europe has nearly half a million telecoms the UK’s largest equipment rental groups is
a challenging winter could put telecoms towers, with most providing battery backups also stockpiling extra diesel generators in
infrastructure to the test. that last around half an hour to run the anticipation of high demand going into winter.
The potential power issues have been mobile antennas. “We’ve been doing contingency planning
fueled by Russia’s invasion of Ukraine, with French electricity distributor Enedis has since the war in Ukraine broke out,” Gary
Russia deciding to halt gas supplies via put forward plans for a ‘worst-case scenario’ Aitkenhead, Equinix’s SVP of Europe, Middle
Europe’s key supply route in the wake of this that will see power cuts lasting up to two East and Africa operations, said.
conflict. hours, affecting different parts of the country “We don’t ever expect to have to run for
Four telecoms executives say that there on a rotational basis, notes Reuters. Any more than a few hours, or at worst case a day,
are currently not enough backup systems blackouts would exclude hospitals, police, on diesel but we’re prepared to run for up to
in many European countries to handle and government facilities. a week.”
widespread power cuts. This potentially Sources claim that the French bit.ly/EuropeanEnergyCrisis

UK government could ration diesel

supplies to keep data centers up
The UK government is considering rationing data centers’ access to diesel
fuel for backup if the power crisis accelerates in coming months.
Government officials have discussed allocating diesel for data center backup
generators if the continuing energy crisis leads to power cuts from the National
Grid, according to Bloomberg.
Data centers make up 2.5 percent of the UK’s electricity demand, and in recent
years have faced increasing opposition from local residents. In West London
it was reported that housing building projects were not being allocated power,
because data centers had already taken the available capacity.
As well as grid power, data centers need backup, currently nearly always
provided by diesel generators. There have been reports of data centers stockpiling
diesel for what is expected to be a difficult winter.
“Our members have taken all necessary precautions by filling up their reserves,
but we need to see government take necessary measures to ensure a continuous
supply in the unlikely event of prolonged blackouts,” Matthew Evans, markets
director at technology industry group techUK, told Bloomberg.
Data center operators in turn are coming forward with suggestions to progress
the idea of using their backup systems to shift demand off the grid at key times, to
avoid the need for blackouts.
bit.ly/DieselSupplyCrunch

8 | DCD Magazine • datacenterdynamics.com

>>CONTENTS DCD Magazine #46

Dominion to resume connecting

new Loudoun data centers, but
capacity still limited
Dominion Energy has found a way to connect some new data centers in
Loudoun County, Northern Virginia - but there will still be some that have
to face delays till 2026.
In July, the energy utility shocked data center operators in the
world’s most concentrated data center hub by announcing that it could
not guarantee new facilities would get power through its network of
overhead lines, meaning planned connections for some facilities in the
East of the County would be delayed for years.
But in late September, Dominion said that it is once again connecting
new data centers. Some planned projects will be held up for as much as
four years, because of constraints in its transmission infrastructure.
Dominion Energy now says it has been working with customers
to spot strategies to squeeze some incremental capacity out for new
projects, according to Data Center Frontier.
“After completing a comprehensive analysis of our system and
accelerating several near-term projects, we’ve been able to lift the
temporary pause and resume new data center service connections on
an incremental basis,” Dominion’s Aaron Ruby said.
bit.ly/AtLeastThatsSomething

“No credible pathway

to 1.5°C in place,” UN
AWS buys 105 back-up diesel generators warns
for new data center in Dublin
The world is heading for a
catastrophe as nations fail to
reduce carbon emissions, the
In response to concerns over impact on electricity networks in the region UN’s environment agency has
warned.
Amazon has applied for an emission license to It was this report that led to the consideration The agency said that there
Ireland’s Environmental Protection Agency (EPC) of a moratorium on data centers country-wide. is ”no credible pathway to 1.5°C
to install 105 diesel generators at its new Dublin Instead, the government published a revised in place” for 2030, and even
data center site. statement in which it said that data centers should reducing emissions to hit 1.8°C
First reported by The Times, the application from make efficient use of the country’s electricity by 2050 seems unlikely.
Amazon requests 105 backup diesel generators grid by using available capacity and alleviating “This report tells us in
and four diesel-powered fire pumps to be located constraints, increase renewable energy use, cold scientific terms what
at the data center site in Clonshaugh Business and colocated with a renewable generation or energy nature has been telling us all
Technology Park. The units will have the ability capability, be decarbonized by design, and provide year through deadly floods,
to general a total of 674MW of power. Industrial opportunities for community engagement. storms, and raging fires:
emissions licenses are required when units are There remains a de facto moratorium enforced we have to stop filling our
expected to generate more than 50MW of power. by the grid in the Dublin area, with EirGrid stating atmosphere with greenhouse
Over the summer Amazon was granted no new grid connection applications for data gases, and stop doing it
planning permission for two new data centers at centers will be accepted in Dublin until 2028. fast,” Inger Andersen, the
the Clonshaugh Business and Technology Park. Interxion (Digital Realty) paused plans for executive director of the UN
Located on the site of a former Ricoh building expansion in the area as a result, while Dataplex Environment Programme
once earmarked for T5’s first European facility, it’s recently entered voluntary liquidation after (UNEP), said.
unclear if Amazon had been previously granted EirGrid denied power contracts at two data center “We had our chance to
a connection by EirGrid before the current sites. Microsoft, AWS, and Equinix have also make incremental changes,
moratorium was brought in. reportedly paused projects in the area. but that time is over.
According to The Times, applications for Ireland’s Commission for Regulation of Only a root-and-branch
licenses have ‘flooded into the agency,’ and nine Utilities has reportedly encouraged data centers transformation of our
of the ten applications received this year are for and other large consumers of energy to turn to economies and societies can
Amazon’s data centers. emergency power generators. Since January of save us from accelerating
The application comes as a response to last year, a total of ten facilities — nine operated climate disaster.”
concerns over the impact data centers are having by Amazon and one by Microsoft — have applied She added “every fraction of
on electricity networks in the country. It was for industrial emissions licenses. K2, Equinix, and a degree matters: to vulnerable
reported earlier this year that the amount of Echelon applied for permits in 2020 and 2021, communities, to ecosystems,
metered electricity consumed by data centers in ahead of the CRU decision. and to every one of us.”
Ireland reached 14 percent in 2021. bit.ly/BackUpALotOfSecs bit.ly/ItWasGoodWhileItLasted

Issue 46 • November 2022 | 9

COMPREHENSIVE ENGINEERING EXCELLENCE
www.anordmardix.com
>>CONTENTS DCD Magazine #46

Intel to lay off a “meaningful number” Oracle lays off more

than 200 California-
of employees in $10bn spending cut, based workers
Oracle has laid off more than

as chip sales crater 200 workers at its Redwood

City location.
The job cuts happened last
Semiconductor designer and manufacturer of wins in the data center and supercomputer
week as the tech giant filed
Intel plans major layoffs and spending cuts space, while issues with Intel’s manufacturing
a Worker Adjustment and
as its internal issues are compounded by a process meant that its products fell behind. At
Retraining Notification in
slowing global economy. the same time, x86 as a whole is under attack
California.
Revenues fell 20 percent year-over-year from Arm processors, as well as RISC-V and
In total 201 jobs have been
in the last quarter, while net profit fell by 85 novel architectures.
cut, said to be ranging from
percent. Intel commanded an 86.1 percent presence
data scientists, application
Other than its networks and Edge division, in the x86 server chip market in the second
developers, marketing
which grew in revenue by 14 percent to $2.3 quarter, down from 90.5 percent in the year
specialists, and software
billion, revenue fell across the board. before - and well down from highs of 98
developers.
PC processors dropped by 17 percent to percent.
Oracle previously
$8.1bn, its relatively new chip foundry business With both internal and external pressures
announced cost-cutting plans
dropped two percent to $171 million, and its cutting into sales and profit, Intel said that it
to save $1 billion, noting that
servers and AI chips fell 27 percent to $4.2bn. would cut costs by $3 billion in fiscal 2023, and
job cuts would be likely. Earlier
But CEO Pat Gelsinger tried to pitch its up to $10bn by 2025.
this year DCD reported that
server chips as a positive sector, saying that its “These savings will be realized through
further job cuts are likely to
upcoming 4th Generation Xeon Scalable chips multiple initiatives to optimize the business,
happen beyond the US, in
“will be our fastest ever Xeon to a million units.” including portfolio cuts, right-sizing of our
Europe, India, and Canada.
The next three generations of Xeon server support organizations, more stringent cost
The layoffs follow Oracle’s
products are all on target, he claimed, and are controls in all aspects of our spending and
$28.3bn acquisition of
“making very good milestones.” The 4th Gen improved sales and marketing efficiency,”
healthcare IT company Cerner
‘Sapphire Rapids’ had been repeatedly delayed David Zinsner, chief financial officer at Intel,
in June.
due to technical issues. said during an earnings call.
Oracle still plans to invest
Irrespective of the economic slowdown, Speaking to Barron’s, Zinsner said there
in its cloud service, as it ramps
Intel’s server business has faced pressure from would be a “meaningful number” of job losses,
up to serve TikTok, which it
a resurgent AMD, which has eaten at its once- echoing earlier reports that the company could
gained as a customer after
near-monopoly market share. AMD’s Epyc be set to fire thousands of employees.
then-President Trump tried to
processors have scored an increasing number bit.ly/AllLayoffsAreMeaningful
ban the Chinese social media
platform, causing it to shift to
the cloud provider founded by
a Trump donor.

bit.ly/CoveringCernerCosts

Microsoft lays off 1,000

employees across company
Microsoft has quietly laid off around 1,000 employees across multiple
divisions of the company.
The true scale of the cuts, first reported by Insider, are not known.
Among the divisions impacted by the layoffs are Xbox, Xbox Cloud,
Microsoft Strategic Missions and Technology organization, Azure, and
Microsoft government. The Mission Expansion cloud government team
is among those potentially on the chopping block.
“Like all companies, we evaluate our business priorities on a regular
basis, and make structural adjustments accordingly,” Microsoft said in
a statement. “We will continue to invest in our business and hire in key
growth areas in the year ahead.”
The company announced it would lay off less than one percent of its
180,000-person workforce in July, and has slowed hiring since May.
A number of high-profile executives have also left Microsoft Azure
over the past year, including its VP of global infrastructure, VP of cloud
infrastructure M&A, CVP of Azure IoT, and VP of global data center
construction.
bit.ly/AQuietFiring

Issue 46 • November 2022 | 11

 >>CONTENTS
Whitespace

Rogers and Shaw

merger deal hits a
snag
The proposed $20
billion merger of Rogers
Communications and Shaw
Communications faces
uncertainty after mediation
with Canada’s competition
bureau failed.
Rogers, Shaw, and
Quebecor failed to mediate
their differences with Canada’s
competition bureau, according
to Reuters. Police raid SK Group over Kakao fire
Rogers first announced its
intentions to buy Shaw in March
outage, as Korea’s government impose
2021, in a move that would
see four Canadian operators
safety drills on data centers
consolidate down to three, Korean police have raided the offices of SK Group, Meanwhile, the country’s Ministry for Science
but the move was met with and the SK data center which caught fire in October. and ICT (MSIT) is stepping in to impose disaster
fierce opposition by Canada’s Local police confiscated documents relating management procedures on private companies
Competition Bureau over fears it to the fire, which brought down the KakaoTalk in Korea. All the country’s large data centers will
would hurt competition. messaging service on Saturday. be subject to a government disaster management
In a bid to mitigate KakaoTalk is used by 90 percent of South system, and subject to regular inspections and
competition concerns, Koreans, and the outage brought down many safety drills, according to Korea JoongAng Daily.
Rogers outlined its plans to finance and travel applications that rely on The measures promised to establish a center
sell Shaw-owned Freedom KakaoTalk IDs. The disruption continued for much for disaster prevention for digital infrastructure,
Mobile to telecoms and media of a week, and the data center is still operating which will address private data centers as well as
firm Quebecor, through its without backup power, so further disruption is government facilities.
subsidiary Videotron for possible at the time of publication. Nationwide inspections of data centers and
CAD$2.9bn (US$2.3bn), which Kakao, which has seen one of its CEOs step network infrastructure will be carried out by MSIT
would allay any fears over the down in the wake of the incident, has blamed SK’s with the National Fire Agency, and an expert
competition in Canada. Li-ion batteries for the fire. group will create a list of measures to improve data
However such proposals The police will also be interrogating SK Group center safety.
have not been enough, with officials, according to Yonhap News Agency. bit.ly/KakaoGoesKaka
Rogers, Shaw, and Quebecor
detailing in a joint statement
that “the mitigation did not
yield a negotiated settlement.”
Peter’s Starlink factoid
The companies plan to try Researchers have reverse-engineered Starlink’s communication
to come to a resolution with a
network to use it as a global positioning system. But to be as
tribunal this November.
bit.ly/DealGetsRogered
accurate as GPS, SpaceX would have to provide more data.

Elon Musk says SpaceX will pay for

Ukraine’s Starlink service after pushback
SpaceX CEO Elon Musk has reversed calls for But in April it was revealed that USAID
the US government to continue to foot the bill spent millions on buying thousands of
for Starlink Internet services in Ukraine. terminals, while the governments of France
The world’s richest man tweeted: “The hell and Poland also acquired thousands of the
with it … even though Starlink is still losing dishes. Some Ukrainian citizens have also
money & other companies are getting billions tweeted that they’ve been paying for Starlink
of taxpayer $, we’ll just keep funding Ukraine out of their own pocket.
govt for free.” But SpaceX was covering some costs, and
The company had long claimed credit for sent a letter to the Pentagon asking for money,
the tens of thousands of Starlink dishes sent to which leaked. This was met by a public outcry,
Ukraine since the outset of Russia’s invasion, exacerbated by recent Musk comments that
and said that it had not received money for echoed the demands of Russia’s government.
the deliveries. bit.ly/AStrangeMan

12 | DCD Magazine • datacenterdynamics.com

>>CONTENTS DCD Magazine #46

Saboteurs cut fiber cables in France, in Shetland outages

caused by submarine
second incident this year cable breakages
The Shetland Islands
Multiple fiber cables were cut in Southern Paris to other locations in France, in three experienced a major outage
France, in what appears to be a targeted attack. directions.” that affected landline,
The incident is the second in the country Arthur PB Laudrain, a researcher at the Internet, and mobile
this year. Back in April, multiple cables were University of Oxford’s department of politics services.
cut in the country overnight in a coordinated and international relations who has been The outage at the
incident. studying the attacks, added: “It implies a lot of archipelago in the Northern
“We are aware of a major cable cut in the coordination and a few teams.” Isles of Scotland was been
South of France that has impacted major No groups or individuals claimed caused by a breakage in the
cables with connectivity to Asia, Europe, US, responsibility for the damage, and French SHEFA-2 submarine cable.
and potentially other parts of the world,” cloud police have not announced any arrests related The cause of the break hasn’t
security company Zscaler said in a blog post. to the damage. It is not known if the cuts are been disclosed.
“As a result of the cable cut, customers may related to Covid conspiracy theories, anti-tech BT blamed the outages
see packet loss and or latency for websites and activity, or for another reason. on breakages to a ‘third-
applications which traverse these impacted Fiber optic cables used for Orange’s network party cable that connects
paths.” were intentionally cut in the Paris region back the Shetland’, which is 100
The company said that there were at least in 2020. miles off the North coast of
three cable cuts - Marseille-Lyon, Marseille- The week before the cut, submarine cables Scotland.
Milano, and Marseille-Barcelona. Police were to the Shetland Islands were severed, causing The police declared the
on-site at the first cut, investigating the cuts, Internet outages in the remote Scottish outages a major incident,
which caused delays to the repairs. archipelago (see right). while the MP for Orkney and
The cuts echo those in April, although it is While such outages are usually accidental, Shetland Alistair Carmichael
not known for certain that they are connected. British tabloids have focused on the presence said that the damage had
The Spring cuts caused outages across France, of a Russian “research ship” near the cut, caused a “catastrophic
impacting 10 Internet and infrastructure suggesting that they could have intentionally impact.”
companies and several cities. damaged the cable. BT Group said in a
“The people knew what they were doing,” Russia is also accused of sabotaging the statement: “Engineers are
Michel Combot, the managing director of Nordstream gas pipeline in the Baltic Sea, working to divert services
the French Telecoms Federation, told Wired. but there is no proof it was involved in the via other routes as soon as
“Those were what we call backbone cables that Shetland cut. possible and we’ll provide
were mostly connecting network service from bit.ly/WillThereBeAThird further updates. Our external
subsea provider is also
looking to restore their link
quickly.”
SHEFA-2 was initially
deployed in 2007.
bit.ly/OhShetNo

Twitter data center brought

down by California’s extreme
heat wave
A Twitter data center was brought offline by California’s extreme heat
wave, as equipment shut down during record temperatures.
The company avoided an outage by transferring its workloads to its
data centers in Atlanta and Portland, but said that if either of them had
failed the social media website would have gone too.
“On September 5th, Twitter experienced the loss of its Sacramento
(SMF) data center region due to extreme weather. The unprecedented
event resulted in the total shutdown of physical equipment in SMF,”
Carrie Fernandez, Twitter’s VP of engineering, said in an internal
message seen by CNN.
“All production changes, including deployments and releases to
mobile platforms, are blocked with the exception of those changes
required to address service continuity or other urgent operational
needs.”
The outage occurred before Elon Musk acquired the site.
bit.ly/WelcomeToOurFuture

Issue 46 • November 2022 | 13

DCD Magazine #46 >>CONTENTS

The last
data center
Long-term data storage
enters a new epoch

Sebastian Moss
Editor-in-Chief

Images by Sebastian Moss

14 | DCD Magazine • datacenterdynamics.com

>>CONTENTS The last
Escape
dataUkraine
center 

B
uried deep in a mountain at the has been lost in wars, fires, and through Hallstatt, Austria, rows of neatly organized
edge of civilization there lies what institutional decay, never to be recovered. ceramic tiles attempt to provide a snapshot
may end up being humanity's last Our understanding of ourselves and our past of our world.
message. is told through what little survived, providing
The most immediately discernable tiles
a murky glimpse that is deeply flawed and
To get there we traveled over are readable to the human eye - with words
relies on the skewed records of kings and
permafrost and up a steep passage, past and images printed onto them at 300 dpi
emperors.
signs warning of polar bears ahead. Then resolution, similar to a normal color printer.
we descended into the dark mines, our Now, things are different. We are flooded
Less visually exciting at a distance, but
headlamps illuminating falling ice crystals with data, from individuals, corporations,
perhaps far more important, are ceramic
disturbed by our presence, bathing us in and machines themselves. But we keep
microfilm plates. Kunze turned to physical
glittering and ephemeral showers. that data primarily on hard drives and solid
vapor deposition, a method of vacuum
state drives, which last mere decades if
It's not clear how long we traveled deposition that produces thin films and
kept unused in ideal conditions, and just a
down the shaft, time travels differently coatings on substrates such as metals,
handful of years if actively run.
underground. Here in this Svalbard and then laser etches data at five lines per
mountain, it is measured in eons, not hours. Other common storage platforms are millimeter. This gives around 500 times the
magnetic tape and optical discs, which density of the original plates - which will
We go past the Global Seed Vault, a
themselves come in multiple formats of be used to store 1,000 of the world’s most
backup facility of the world's seeds in case of
varying density and lifespan, but are often important books.
disaster, and journey further downwards. At
used for ‘cold’ longer-term storage.
last, we come to a door, glowing in the pitch- This, in turn, sparked a government-
black. Emblazoned on it are the words "Arctic All have their uses and individual benefits funded project to build a femtosecond
World Archive: Protecting World Memory." and drawbacks, but the simple fact is that laser, which could write onto even thinner
if we stopped transferring data to new materials such as glass-ceramic. "It's very
Before we talk about what's behind that
equipment, nearly all of it would be gone early days, but we have proved that it's
door, we should understand the two great
before the century is out. possible to write and read 10 gigabytes per
data challenges it hopes to solve. It is joined
second" at much higher density, he said.
by dozens of startups, researchers, and even "The only written records of our time
a trillion-dollar corporation in competing to would be the embossing on stainless steel As for its lifetime, it should "far exceed the
figure out the future of data. cooking pots saying 'made in China' and existence of our Solar System, so you could
probably the company logos on ceramics," say it's eternal," he said. The technology
One challenge is philosophical, that of
Memory of Mankind (MoM) founder Martin is being developed by a company Kunze
our death and destruction, and what we
Kunze explained. cofounded, Cerabyte, currently still in stealth
leave behind. The other is more immediate,
mode.
that of growing hordes of data, which Kunze is one of a select few hoping to
threaten to overflow our current systems and prevent such a tragic loss for our future. To Cerabyte does not expect to produce the
leave us unable to keep critical data in an do so, he is looking to our past. tech on its own, and has turned to Sony -
economic way. which has an optical disc factory in Austria
that is slowly declining with the death of the
A collection of memories format - to potentially develop "a minimum
The beginning of recorded viable product that we aim to have in one
"I studied art with a focus on silicate
knowledge and a half years," he said.
technologies and ceramics," he said. "The
The story of data is an ancient one. Some idea of using ceramics as a data carrier is not He's far from alone in trying to reinvent
73,000 years ago in what is now South new, it's 5,000 years old." the data storage landscape. While Kunze
Africa, an early human picked up a piece of turned to the past for inspiration, others have
Like the archive in Svalbard, and the Deep gone for an approach that appears ripped
ocher and scratched a symbol into a shard of
Sea Scrolls of the past, he has also turned to from science fiction.
stone, in what is our earliest recorded piece
depositing data underground as a method of
of human artwork.
long-term storage.
It took the majority of our species'
2km deep, inside the world's oldest salt The data soup
history to get to written recordings, with
mine beneath the Plassen mountain in
the Kish tablet in 3500–3200 BC, where "I guess you could say we're in the business
humans etched pictographic inscriptions
into limestone. Even then, it took thousands
of years to advance to clay tablets, and still
further to get to papyrus, parchment, and
finally paper.
"We're manipulating DNA for the purpose of both
Most of what happened in the world was
storage and compute and making some real
not recorded. Of what was, the majority progress here"

Issue 46 • November 2022 | 15

DCD Magazine #46 >>CONTENTS

of trying to build a bio-computer," Dave Catalog believes that those researchers, as decoding."
Turek, chief technology officer at Catalog, well as some rival DNA storage companies,
said. "We're manipulating DNA for the This is because the genome industry
have made a crucial error. "You then have to
purpose of both storage and compute and has prioritized accuracy at the base level
solve a fundamental problem, which is that
making some real progress here." over speed, at a fidelity that Catalog doesn't
every time you add another base to your
actually need. "We've got to resolve the issue
synthetic piece of DNA, it takes 30 seconds.
Turek knows all about the intricacies of rapid decoding in parallel to the velocity
So if the cost is 30 seconds for every two bits
of traditional computing - the last time we with which we can write,” he said.
of information, that's not going to work very
spoke was after the launch of the Summit
well." "We have some partnerships established

supercomputer, then the world's most to begin to try to do some real innovation
The company has decided to scale back
powerful, a project he spearheaded during on the read side as well. We have ideas today
and not work at the base pair level, instead
his nearly 23 years at IBM. that we think could easily generate two
constructing an alphabet composed of small
orders of magnitude improvement in speed."
Wetware computing is a new avenue. "I'm snippets of DNA. Catalog now has 100 of
not a molecular biologist, so we're on even these oligonucleotides, as they are known, Even at the slightly lower storage ability
ground here," he joked before launching into with which it can create data by connecting of oligonucleotides, the amount that DNA
a dense explanation of DNA. them together in what is called ligation. can theoretically record is mindblowing.
"You could put all the information in the
The classic double helix construct of "And the machine that we invented
world in this," Turek said, holding up a Pepsi
DNA is made of ladders that are formed of and developed automates that process,
can. Such a feat is still a way off, he admitted,
just four bases: Adenine, Guanine, Cytosine, which is a big deal,” Turek said. “It uses
but the company was able to store all of
and Thymine. With that simple starting inkjet printheads that contain these
Wikipedia into a few droplets.
point, all of the sequences of bases that oligonucleotides. And each reservoir is
dictate everything that makes every living unique from every other one. So that As for the long-term storage abilities, the
thing unique are found. "It's biology's way of each of the nozzles can be instructed to death of a mammoth some 1.2 million years
encoding information," Turek said. fire a particular snippet of DNA out of my ago gives an insight into its longevity. Last
alphabet." year, researchers successfully sequenced the
We have DNA of about 3.5 billion bases previously unknown Krestovka mammoth,
long, which contains all that we are, hinting The ‘Shannon’ machine has three
and its body was not exactly kept in ideal
at the storage potential that could be thousand nozzles that deposit DNA ink drops
conditions.
harnessed. It's not a new idea: Early attempts at the picoliter level, 500,000 times a second.
date back to 1988, while American geneticist These mix together to create a long string Perhaps more enticing still is another
George Church encoded a book into DNA of data holding DNA. “The way it's currently idea being cooked up at Catalog - using
in 2012. configured, I can create a trillion unique DNA for compute. The concept is in its early
molecules in a day,” he said. stages, and only works for very specific types
Those approaches took each of the letters of compute, but would still be a profound
A, G, C, and T, and assigned bit values to This can then be read with a DNA
advancement in computing.
them, potentially allowing incredible data sequencer, with the company currently
storage density. "Immediately, people started using Oxford Nanopore machines. These are “We're doing one case that is inspired by
seeing DNA as the remedy to the overflow much slower than the writing machines - "if a real potential user,” Turek said. “This is not
of information created in modern society," I take 10 minutes to write a whole bunch of a theoretical abstract academic exercise that
Turek said. "And they're completely wrong." data, it might take me a week to do all the originated in a textbook. And we're using

16 | DCD Magazine • datacenterdynamics.com

>>CONTENTS The last
Escape
dataUkraine
center 

“We have turned to warehouse-style robotics,

and we have these little robots that operate
independently, they can move up and down and
along the structure in this crabbing motion"

that customer to guide us in terms of the world are stranger - and they can still sell up, asking what storage should look like,” he
nature of the algorithms, and the other kinds their computers.” recalled, with Microsoft deciding to build
of things that need to manifest themselves upon silica data storage research from the
Nonetheless, Catalog still has a long way
in DNA.” University of Southampton.
to go before it can convince companies
It’ll be a long time before such efforts will to put their data lakes into data soups, and It has gone through multiple iterations
bear fruit, but Catalog is hopeful that the embrace unconventional storage solutions. over the past few years, slowly getting denser
opportunity is vast. “If you want to build a Here, rival Microsoft has an advantage - it as well as easier to read and write. “If we were
parallel computer, which I did for 25 years, can deploy its long-term storage concept to fill our 12-centimeter by 12-centimeter
and you want to add an incremental unit in its data centers, and rely on a robust sales reference platter entirely with data, we’d be at
of computing, it typically has a pretty hefty network to convince users that its approach around five terabytes,” Rowstron said.
cost to it, and consumes a lot of energy and is the way to go.
space,” Turek said.
Halls of glass
In DNA, he argues, “I can make it parallel
In Cambridge, the hyperscaler’s researchers
cheaply to an extraordinarily large degree.
have been experimenting with fused
If you say ‘I want to I want to run this
silica, where a femtosecond laser encodes
instruction 100,000 times in parallel,’ I would
data in glass by creating layers of three-
come back to you and say, ‘100,000? Why
dimensional nanoscale gratings and
not 10 million? Why not 10 trillion?’”
deformations at various depths and angles.
The idea is different, to say the least.
“To read it, we image it with a
We’re used to thinking about the computing
microscope,” Microsoft Research’s
world in terms of electricity and physics,
distinguished engineer and deputy lab
in bits and bytes. “But look at how people
director Dr. Ant Rowstron explained. “It's
are moving on from von Neumann
got these layers in it, and you focus on a
architectures, and beginning to create
layer. And then we take several images
quantum computers,” Turek countered.
concurrently. We don’t spin it, we read an
“We think that we're in the right place at entire sector at a time.”
the right time, because there is a de facto
Project Silica was set up by Rowstron
acceptance of alternative architectures in
and Microsoft after realizing that
the computing world. However strange you
conventional storage was set to hit a
think this might be, the guys in the quantum
bottleneck. “We began from the ground

Issue 46 • November 2022 | 17

DCD Magazine #46 >>CONTENTS

Writing is still slow and expensive, stored the Universal Declaration of Human
requiring high-powered lasers to accurately Rights, Newton’s Opticks, the Magna Carta,
etch the glass in just the right part. Critics, and the King James Bible on small discs.
including some working on the other
Credit: Microsoft “My hope is that in 200 years’ time there
projects in this piece, worry that there are
will be a new storage technology that is even
fundamental limits to how fast you can pump
writing the data onto the blocks of glass, on more efficient, even denser, even longer life,
energy into the glass without causing issues.
the other a reader ready to uncover what is and people are going to say ‘we don't need
“It's been a lot of work,” Rowstron within each block. Other than the robots and to use glass anymore,’” Rowstron said. “But
admitted. “When we first saw the technology the writer, it wouldn’t need any power, and it they’ll move formats because they want to,
at Southampton, it was taking hundreds won’t need any cooling. not because they have to.”
of pulses to write data into glass. But
That means that the first data centers Rowstron believes Silica will prove useful
we’ve been working on how to form these
Silica will inhibit will be massively over- for both major challenges of data storage.
structures with a very, very small number of
engineered. “I guess one day we'll end up "You want to make sure that whatever else
pulses.”
with buildings or data centers dedicated to happens, data from the world is not lost," he
He declined to disclose exactly how far just storing that preserved glass,” Rowstron said. But the tech is naturally focused on the
the project had come, but added “if you said. more immediately pressing concerns faced
compare it to the state of the art, we were by businesses.
significantly below [that number of pulses]. That glass can last around 10,000 years.
“No one's asked us to go further,” Rowstron "Hard drives are languishing, we've had
You can think of it as dollars per megabyte
said. “There are things you could trade-off so little capacity increases in the last five
writing, and I would say the technology is
- you could trade density for lifetime and years; tape is suffering," he said. "There is
now in a good spot.”
things like that. But you've got to remember more and more data being produced, and
Microsoft is also thinking about how our goal is to get a technology that will allow trying to store that sustainably is a challenge
it would be read in a data center. “We us to use this in our data centers. No data for humanity."
have turned to warehouse-style robotics,” center is going to exist for 10,000 years.”
Cold storage that doesn't require constant
Rowstron said. “And we have these little
Others could go further, should there be power would be a huge boost for the
robots that operate independently, they can
demand. The original Southampton work environment, as would moving away from
move up and down and along the structure
found that its much-slower-to-write silica rare earth metals such as those found in
in this crabbing motion, which is pretty
could last 13.8 billion years at temperatures HDDs (of which there may not be enough
cool.”
of up to 190°C (375°F). The researchers there to meet future demand). It is also much
On one end there would be a machine

The guardians In popular culture, the Library of Alexandria - one of matched with moments of renewal or of rescue,"
the greatest repositories of knowledge in antiquity Ovenden said.
of knowledge - was lost to the flames, its endless rows of papyrus
"That certainly is the case today, both in terms of
consumed by fire. Scientific theories, fantastical tales,
communities rallying around their libraries to protect
and bureaucratic minutia became embers in a matter
them, and in library's ability to adapt to the various
of hours.
challenges posed by the big tech companies today."
But this is likely not true, Richard Ovenden, the
We live in a strange time where we have more
head of the Bodleian Library, said. Its death was slow,
access to information than ever, but it is through
as papyrus became less relevant with new technology
the lens of corporations who put profit over fighting
- parchment - and no transition was made, like with
misinformation or loss. "Preservation is not in Google's
the Library of Pergamum.
mission," Ovenden said.
More profound was an institutional decay, as a lack
"We need places on the Internet where information
of management, investment, and care led to its tragic
can be relied on, and I think many of those are
demise.
libraries and archives that are not profit-making
"I think we are in a moment in time where that risk companies."
is high again," Ovenden, author of Burning the Books:
Preserving the creations of corporations, with
A History of Knowledge Under Attack, said. "You
their platforms and proprietary information, is also a
certainly see that with the failure of public libraries in
difficulty.
many Western countries."
"As those preserved worlds become much more
Our story on how we store data for years to come
complex and exist much more in real-time, that task
has focused on the technology, but it's important not
of preservation becomes much more complex and
to forget the humans working to protect and share
philosophically challenging, as well as financially
knowledge.
challenging for organizations," he said.
Librarians and archivists have passed the torch of
"There should be a link with regulation and
knowledge from one generation to the next, ensuring
taxation on the profits of the major technology
the light of humanity does not go out.
companies in order to allow third-party preservation
"Throughout history, there have been moments to take place by not-for-profit preservation
of destruction of knowledge, but they've often been organizations."

18 | DCD Magazine • datacenterdynamics.com

>>CONTENTS The last
Escape
dataUkraine
center 

cheaper over the longer term, when all you hoping to find one that would last a long Anyone can pay to store their data
have to worry about is where to put the glass. time, and store a lot of data. Some last up to at the archive, but it is primarily finding
500 years, but don’t have much density, and business with governments and public
The technology caught the eye of Guy
are dangerously flammable. bodies. Microsoft’s software collaboration
Holmes, CEO of Tape Ark, an Australian
platform GitHub also stored 6,000 software
company focused on moving aging tape “So we embarked on developing our own
repositories on the site (a backup is also
libraries to the cloud. "I've actually got it just film, Piql Film, which is a nano silver halide
stored at the Bodleian library, see box out).
sitting here in my office," he said, grabbing film on a polyester base,” Bjerkestrand said.
what at first looked like a clear square of “It's for people that want to bring their
The company takes binary code and
glass, but revealed rows of microscopic valuable, irreplaceable information into the
converts it into grey pixels, for a total
etchings when a light was shined into it. future to the next generation,” Bjerkestrand
capacity of 120GB per roll of film. Under
said, but added that the company was not
"It's very early days, but it seems to have good conditions, that film can last around
planning to be a curator in and of itself. “That
legs internally there, and it appears to be 1,000 years, Piql claims.
has been a serious discussion, and we came
picking up its density," he said. "I spent a lot
The company has split its data archiving to the conclusion, that no, why should we
of time with the guys at the program just
in two. One is more traditional, where it have an opinion on what's worth bringing
talking through density, access times, access
works with companies like Yotta in India into the future?”
frequency, etc. Customers tend to do one
to convert data to film and store them in
restore per week at the moment based on When pressed on how the AWA will
commercial data centers around the world
backups. As the tapes age, the number of last 1,000 years or more when tied to a
- with Piql highlighting its use against
access requests reduce significantly. corporation, Bjerkestrand said that the
ransomware or data center disasters.
company planned to spin it off as a non-
“We're finding there are these archives of
Then there’s the Arctic World Archive, profit foundation.
infinite retention or tapes that have never
pitched as a more humanitarian mission to
been accessed since they were created.” Currently, however, people pay for
preserve crucial data for future generations.
storage for up to 100 years, after which
His company remains excited about the
Due to its distance from other land the data is sent back to them if they don’t
technology, but is currently focused on the
masses, the fact that it has been declared keep paying. In an eventuality where
more immediate challenge of simply getting
demilitarized by 42 nations, and its lack the institution that first contracted Piql
data off of antiquated tape systems and onto
of valuable resources, the hope is that collapsed, that could cause problems: "They
the cloud, including Microsoft Azure. There,
Svalbard is unlikely to be nuked in any future need to make sure that somebody gets the
it might even go back on tape, but first
conflicts. However, just like everywhere else right to the reel,” he said.
Holmes recommends trying to see if there is
on the planet, the Norwegian archipelago
hidden value in the data. A foundation, on the other hand,
cannot claim total safety - it is one of the
would have a longer-term focus. “Such
"The number of people that don't even fastest-warming places in the world due to
a foundation would have interest across
know what's on their tapes is pretty startling," climate change, and the territory has faced
the world for organizations to support,
he said. "On there could be a cure for a increased aggression from Russia.
it's basically supporting world memory to
disease, but nobody knows. So we get pretty
Still, the land is remote and its empty survive into the future,” Bjerkestrand said. “So
excited by some of the projects we do."
mines are of little use to would-be invaders. I think it's a good cause that you could get
Other efforts have immediate clear value. At the depositing ceremony we attended, sponsors, donors, and supporters for.”
"We got to work with Steven Spielberg on works of art from national archives around
When thinking about scale in that way,
six petabytes of Holocaust survivor videos, the world were placed in storage (and images
you cannot assume that future generations
which needed long-term preservation, it of an Indian couple’s wedding, who paid to
will be able to understand how data storage
needed immutability so that nobody could have “their love recorded for eternity”). Large
formats work. Here, Piql has an advantage:
go and do deep fakes on a holocaust video reels of film are carefully vacuum sealed
As it’s film, you can simply hold it up to the
and say that it didn't occur," he said. The and then placed in the shipping container
light, and the first few frames are pictures of
work required moving critical data off of 300m underground, nestled deep in the
how to access the information.
both tape and film, in an effort to give it a permafrost.
longer life. That data then needs to be able to be
readable with what is available on the roll. “We
With film, we come back to the door in
have a fundamental principle that it should be
Svalbard. The company behind the project
self-contained,” Bjerkestrand said. “We don't
has its roots in the cinema industry, selling
compromise that it should be open source
film projectors and digital light modulation
license and free to retrieve the data with the
technologies to Hollywood and Bollywood.
tools that are on the data. We had scenarios
“And then that movie Avatar came out, where we could do more data on the film, but
and there was a super fast transition from a then it would be too complex to read.”
film-based world to digital projector-based
Simply recording reality and putting it
world,” Piql CEO Rune Bjerkestrand said.
on one of the above media doesn’t work
if future civilizations can’t understand it,
posing a critical challenge for much of the
The return of film data of our day.
While film appeared to be on its way out,
the company was convinced that there was
still something to the medium. Piql studied The death of tools
different types of film from around the world,
“My biggest concern is the loss of knowledge

Issue 46 • November 2022 | 19

DCD Magazine #46 >>CONTENTS

of the software that's needed to quickly of humanity - where data was set to be Layers of control
interpret digitized content,” Vint Cerf, TCP/IP shared and accessible by the world, but was
co-creator, said. transient and easily lost. This, he hoped, gave As the web becomes more centralized and
an opportunity for a new kind of library. in the hands of a few cloud providers, their
“An increasing amount of digital content terms of service risk controlling which
that we create was made by software, which "As we're coming to a change in media information lasts long enough to be stored
is needed to correctly understand, render, type, can we go and start a library, right for greater timescales. At the same time, as
and interact with data, like with spreadsheets away?" he wondered, launching the non- systems get built on top of cloud providers,
and video games where you actually need profit Internet Archive, which tries to create or platforms like Facebook, storing them
a piece of software, plus a bunch of data in a long-term copy of websites, music, movies, would require also recording all that backend
order to exercise it. If you don't have that books, and more. software that is not shared.
software running anymore on the platforms
"The goal of the Internet Archive is to Kahle is less worried about the longevity
that are available 100 years from now, then
try to build the Library of Alexandria for the of data storage devices, and more about
you won't be able to do that.”
digital age," he said. And, so far, the effort ensuring immediate access in an open world.
Cerf, known as one of the fathers of the has been wildly successful: "We're a small
Internet, told DCD that he was “worried organization, we're $20-25 million a year in “100 years ago, microfilm was a new
about the kinds of software that's needed operational costs, and yet we're the 300th technology, and it was greeted with this
to interact with databases, for instance, most popular website in the world." fanfare that we'd be able to make it available
timesheets and other kinds of complex so that people in rural areas could have
But its continued success is threatened by access to information just like the people in
objects, where we may not have as
a changing world. big universities,” he said. “Well, that didn't
widespread implementations available, some
really come about, it ended up just being
of them may even be proprietary.” “I could only start the Internet Archive
used to just reinforce the power structures
after we had gotten the Internet and the
Fellow Internet Hall of Fame member and the publishers that existed at the time.”
World Wide Web to really work, both are
Brewster Kahle shares the concern about
open systems. So if we go into a period The question of world memory is less
specific pieces of software and data being
where the idea of public education or about the medium it is stored on, and more
proprietary or in the hands of corporations.
universal access to all knowledge starts to about how it is used when it is on that
"If you look at the history of libraries, they eclipse, we're in trouble,” Kahle said. medium, he argued. “Microfilm may last 500
are destroyed or they're strangled such that years… if you don't throw it away. But it turns
“We see that now within corporate
they're left irrelevant,” he said. “And that used out, people will just throw it away even if it
environments, and we're starting to see it
to be by king and churches, but these days it hasn't been copied forward.
in government environments, whether it's
is governments and corporations.”
banning books or where you can pressure “We need not just formats that will last
As one of the developers of the World organizations to do things without going a long time, we need to keep the material
Wide Web precursor the WAIS system, through the rule of law, but you go through in conversation, in use so that people will
Kahle found himself at an inflection point the rule of contract.” continue to love it and keep it going.” 

Our obituary
We don’t know when humanity will end. It could be soon, with Lomberg’s anatomically correct drawings of a man and woman
climate change, nuclear war, or another pandemic. Perhaps we will were not included on the Golden Record.
survive, eventually spreading to distant stars.
Another image that was not included was that of an atomic
When we do, we may come across a golden disc floating through bomb or mushroom cloud, despite the pervading threat of nuclear
space, carrying a message from our past. Or, possibly, someone else holocaust when the vessels launched in 1977. “Carl [Sagan] didn't
might come across it. want it to seem like a threat,” Lomberg said. “He didn't want anything
on record that could seem like ‘if you mess with us, this is what we
“There were two audiences we designed for - one was the
could do.’ He wanted to greet the cosmos with open arms.”
extraterrestrial audience, and that was the one that I was most
concerned with, and the other was the message to ourselves,” Jon The discs contain music from around the world (which is
Lomberg, NASA's design director for the Golden Record, recalled. available online for those curious), as well as different languages,
all of which will last an incomprehensibly long time. The side facing
“In a sense, it was a message to ourselves, as well as a message
outwards is covered by a box that “lasts about a billion years, and
to extraterrestrials saying this is a snapshot that one group of people
then it's about another billion years to erode the outward facing
thought would be a good capture of the Earth at this point in time.”
side,” Lomberg said. “The inward-facing side lasts a lot longer - they
While destined for the stars aboard the Voyager spacecraft - think up into the trillions of years."
which are the furthest man-made objects from our planet - the team
Should it be found by another species billions upon billions of
had to contend with very terrestrial challenges.
years from now, it will be our last message to the cosmos. And, by
First, there was copyright, with some songs not allowed on the dint of data storage limitations and intentional curation, it will be an
golden vinyl. imperfect one.

Then there were issues with nudity, after the earlier Pioneer plaque But Lomberg is happy with that: “In a sense, this is our obituary;
which included a line drawing of a naked man and woman drew let's be remembered for Mozart, not Hitler. We'll likely be done in
criticism from the more prudish for showing a penis, and those on the by our own flaws, our own shortcomings may well destroy us, and
other end of the spectrum for the censorship of female genitalia. that's punishment enough. But that wasn't the whole story of us.”

20 | DCD Magazine • datacenterdynamics.com

 Sponsored by

Next-Gen Networks
Supplement

Connecting the world

INSIDE

How peering went global Building light into chips A second look

> DE-CIX’s new CEO reflects on > Can we use photons for compute > Ending the leap second and
the growth of interconnection and networking? saving time from itself
IT professionals get more

when they operate at the edge.

EcoStruxure™ Micro Data Center from

Schneider Electric™ lets you operate in
any edge environment.
• Enjoy fewer costly service visits.
• Deploy faster with pre-integrated solutions delivered
in shock packaging.
• Rely on the support of our partner and services network.

#CertaintyInAConnectedWorld
EcoStruxure 6U Wall Mount
IT Expert EcoStruxure
apc.com/edge Micro Data Center

©2022 Schneider Electric. All Rights Reserved. Schneider Electric | Life Is On and EcoStruxure are trademarks
and the property of Schneider Electric SE, its subsidiaries, and affiliated companies. 998-22321004_GMA
22 | DCD Magazine • datacenterdynamics.com
 Next-Gen Networks 

Sponsored by

Contents
24. How peering went global
Changes in the world’s networks turned
The networking challenge

Y
Germany’s Internet peering platform into a
global phenomenon ou know the story. further into the data center (p28).
The amount of That could mean processing at
28. Building light into chips data the world the speed of light, just in time for an
produces is growing explosion in artificial intelligence
The AI revolution needs so much
exponentially. workloads.
processing power, could it be time to More importantly, “We have about a 42x
bring in a new generation of chips that the data is created from more improvement in latency, because
combine photonics with conventional sources than ever before, and then the processing is happening at the
silicon? has to travel to yet more end points speed of light," CEO Nick Harris
than ever. said.
31. A
 second look "You're doing multiplication and
Hyperscalers want to end the leap second, Gotta connect them all addition as light flies through the
as things start to turn negative The story of how the data center chip."
sector linked up cannot be told
without looking at DE-CIX (p24). Ain't got time for that
We talk to the CEO of the There are 31,536,000 seconds in
German Internet exchange a year. It takes just one of them to
company about how it became one break everything.
of the key lynchpins of our modern Since 1972, the world has added
world. leap seconds to UTC every few
“When I’m asked 'what's the years in an effort to bridge the gap
difference between DE-CIX and between atomic clock-based time
Equinix?' I say 'we have more data and the time derived from the Sun's
centers,'" CEO Ivo Ivanov told us. position as our spin decelerates.
It doesn't actually own any of This may not sound like a big
those data centers, of course. "We deal, but every time it happens is a
are in more than 700 facilities calamity for networks. Computer
around the globe," he said. systems that rely on a precise and
"Including Equinix.” identical time cannot handle the
Looking ahead, it represents an change, crashing spectacularly.
opportunity for an open peering Companies like Google
network, where these networks are and Meta have come up with
not controlled by a single company. workarounds, smearing a second
This could have a profound over hours. This is a step forward,

24 impact on the next iteration of the

Internet, especially as Edge comes
to the fore.
but still carries a great risk.
That danger is only growing.
The Earth's spin is no longer
slowing, in fact it is speeding up.
Let there be light This change means that we will
Over fiber, data travels at the speed soon have to do a world first and
of light. Then it reaches the data roll out the negative leap second.
center. With systems crashing even
We're used to compute being with a time addition that we're used
in the realm of electrons, but that to, how will they react to such an
means heat and latency. event?
Lightmatter hopes to use Companies don't want to find

28 31 photons for computing and

interconnection, bringing light
out, so suggest killing off leap
seconds entirely (p31).

Next-Gen Networks Supplement | 23

 Next-Gen Networks Supplement >>CONTENTS

How peering went

global Peter Judge
Executive Editor

Changes in the world’s networks turned

Germany’s Internet peering platform into a
global phenomenon

I
f your business wants a connection instance now connects more than 3,000
in Africa, India, or the Far East, networks.
you might find yourself looking
“In 1995, DE-CIX was established as a
at a platform originally founded
as a very local service handling pure peering point, a fabric operator,” says
Internet traffic in Germany. Ivo Ivanov, DE-CIX’s CEO.

Three ISPs launched the German

The story of how DE-CIX changed
IXP in the back room of a post office in
from a single-country Internet exchange
Frankfurt, and expanded into an Interxion
to a global interconnection player is the
data center, becoming one of the largest
story of big changes in networking -
Internet exchanges in Europe.
and in the way society consumes those
networks. “Over time we grew in Frankfurt,
and we added Hamburg and Munich,”
he says. “In 2007, Telegeography said
we were a new telecommunications
Starting in a post office capital of Europe, because of the huge
concentration of traffic between the
The Internet was created in the United
Western and Eastern hemispheres.”
States. By the early 1990s, it had grown
internationally, linking networks round That’s when DE-CIX realized it had
the world. But all peering still took an international opportunity. If you’ve
place in the United States, so when two created a distributed carrier-neutral
Internet users on different networks exchange platform in multiple data
communicated, all their traffic still had to centers in one metro area, then you can
go through the US. Local groups could see an answer. In do it in others.
late 1994, a group of five British Internet
This applied in every country in the service providers (ISPs) set up the London “A data center- and carrier-neutral,
world, creating heavy delays, and massive Internet Exchange (LINX), using a Cisco distributed platform present in a metro in
network costs in days when international Catalyst switch with eight 10Mbps ports in different data centers, all interconnected
bandwidth was much more limited and London’s Docklands. into one integrated fabric, is great from
expensive than it is today. the redundancy point of view, and from
In 1995, Germany followed suit swiftly, the distribution point of view, with a very
Two users connecting in London or with its own Internet exchange point high level of accessibility.”
Frankfurt had to effectively communicate
(IXP), the Deutsche Commercial Internet
over a two-way link to the only Internet The nature of an IXP began to look
Exchange (DE-CIX).
exchange points, with all their traffic more like a business model than a simple
going to the US and back, adding massive Internet exchanges in all countries network exchange, he says: “Potential
delays and telecoms costs. have grown massively: DE-CIX for participants can join the fabric regardless

24 DCD Supplement • datacenterdynamics.com

>>CONTENTS Peering into the future 

in the US cities of New York, Dallas,

Chicago, Richmond, and Phoenix.

Alongside that, there are now

European exchanges in Marseille,
Palermo, Lisbon, and Barcelona, with
another serving the Turkish market in
Istanbul. Five years back, an exchange
was established in Mumbai, with other
hubs following in Delhi, Kolkata, and
Chennai.

DE-CIX’s Indian operation claims to

be the largest interconnection platform
based on the numbers of participants in
Asia, Ivanov says.

Southeast Asia followed with IXPs

in Malaysia, and Singapore. There’s a
partnership announced with the Getafix
exchange in the Philippines - and a move
to offer exchange services across the
whole of Africa.
of which data center they're physically
located in. It's a matter of cross-connects The African service takes a new
only.” network approach, matching a more
consumer-centric network market. It’s a
That’s pretty similar to the Equinix
managed solution that Ivanov calls “DE-
International Business Exchange
CIX-as-a-service” or ‘interconnection
(IBX) model, he says, but with one big
fabric in a box.’
difference: “They [Equinix] offer their
fabric to their own customers only.” “We deliver a turnkey full-fledged
interconnection fabric with hardware,
“Our success in Frankfurt gave us the
software, and processes - but also
confidence to export this type of know-
including marketing and business
how, this experience on the engineering
development activities,” he says. “It's
and business development side, to other
literally the full cycle needed to run a
regions.”
successful interconnection fabric and
DE-CIX looked for regions that lacked create an ecosystem.”
this type of infrastructure or ecosystem,
Coming back to the comparison
where there was no localization of
with Equinix, he says: “When I’m asked
content and data gravity. Its first target
'what's the difference between DE-CIX
was Dubai.
and Equinix?' I say 'we have more data
‘UAE-IX powered by DE-CIX’ was centers.'"
launched with local telco Datamena
Of course, DE-CIX doesn’t own data
in 2012. Since then, it’s become an
centers, he explains: “We do not have
international and regional hub, and
data center operations. We collaborate
Ivanov reckons the presence of DE-CIX
with the data center operators. We have
was “one of the main factors” contributing
more data centers enabled. We are in
to Dubai’s ascendence in digital
more than 700 facilities around the
commerce in the Middle East.
globe. Including Equinix.”
“To create an attractive framework for
The service has expanded up the
operators who are not from the region,
stack as well, offering cloud connectivity
the regulatory authority created the so-
over and above network peering, with a
called free interconnection zone based on
so-called “multi-service access port,” that
our guidance.”
can support what DE-CIX calls a “cloud
router,” designed to let customers hook
together multi-cloud networks.
Global networks are changing That means DE-CIX can offer a
network security service, giving a direct
With that success, DE-CIX next went into
Layer 2 network connection into cloud
areas that already had infrastructure, but
services such as Microsoft applications
wanted to consume networks differently.
like Azure, Office 365, CRM, and ERP.
The German outfit set up exchanges

Next-Gen
Automation Supplement || 25
Networks Supplement 25
 Next-Gen Networks Supplement >>CONTENTS

"We see a huge wave of new market

participants on platforms like DE-CIX,
who are probably not interested
in peering directly"

Enterprises are network Ivanov thinks that banks and

automotive companies will be the next
operators now
big global network operators: “We know
These services are aimed at a different set from different projects which have been
of customers from the traditional ISPs and announced that finance institutions have
network providers linking to an Internet started investing in their own subsea
exchange port. They are picked up by cables, and automotive companies are
large enterprises in multiple industries. building their own mobile networks. If evolve even further in the future, have
you control the data journey, you control
“We see a huge wave of new market one thing in common - they're extremely
the gateway, and you can control the
participants on platforms like DE- performance sensitive. That means
gateway only if you are directly involved
CIX, who are probably not interested they are latency sensitive. Every single
in infrastructure.”
in peering directly,” he says. “They're millisecond counts.”
interested in cloud connectivity.” What these customers want is virtual
He goes on: “All the crucial processes
private networks - and that’s something
All the big industries are getting in a company are somehow related to
a peering platform can offer, by setting
involved in direct management of their real time communication, and latency is
up a closed user group. This means, for
connectivity, says Ivanov. “And the bigger an extremely important factor. I love to
instance, that an auto firm can keep tabs
they are, the higher the demand for global say latency is the new currency in our
on all the data that flows in and out of a
network presence.” connected car. industry.”

For instance, he says, eight years ago “The closed user group approach gives As-a-service exchanges can be set up
Netflix and Apple didn’t own a single them a private virtual ecosystem for direct anywhere, he says: “We’re creating new
Point of Presence (PoP). “They were interconnection, which is more isolated, telecommunications hubs in the Edge, in
not connected to even one Internet more secure and in specific compliance tier two and tier three. We have projects
Exchange. Today. Netflix and Apple have with regulatory requirements.” for creating Edge interconnection setups
hundreds of PoPs in hundreds of different on highway crossroads or next to a mobile
exchanges around the globe. cell tower.”

“They're still huge enterprises,” he says. As with the large facilities, DE-CIX
Peering at the Edge
“But today they are also a global network isn’t making its own Edge facilities. It’s
operator. They run their own global The neutral exchange approach also placing a “pizza-box” of its own kit into
network for resilience, for cost saving, but lends itself to Edge services, he says: “As the appropriate Edge modules - or any
even more importantly to control the data we all know, all the applications which other configuration up to four cabinets in
journey to their customers.” are extremely important today, and will a Tier One hub.

26 DCD Supplement • datacenterdynamics.com

>>CONTENTS Peering into the future 

higher the need for redundancy so they

can be extremely solid in the case of an
outage.”

This sort of redundancy might even be

required by regulators which are starting
to require financial sector organizations
to have a plan to mitigate the risks of
cloud concentration.

These regulations will require

organizations to have a multi-cloud
strategy to avoid relying on one single
cloud service provider, he says: “But it also
means ideally that you physically connect
to different cloud zones and to different
cloud connectivity solutions as well, for
redundancy.”

It’s another example of an opportunity

emerging from the combination of
economic reality and network technology.
In the end, it’s the basic changes in the
way we use networks that have turned one
German peering organization into a global
network platform. 

In the US, DE-CIX is working with Edge this can be a good thing.
players like DartPoints, while Ivanov says
In terms of network peering, there
some interesting options are emerging:
are other players offering software-
“AtlasEdge is an interesting one in Europe,
defined networking (SDN) services, such
and I believe companies like EdgeConnect
as Megaport, Packet Fabric, and Console
have started looking into a related
Connect. But he’s happy to collaborate
approach - so I think the landscape will be
with them to extend the reach of both
very interesting.”
services, and he argues that overlapping
services are complementary, and can
reduce the inherent risk of moving to one
The joy of duplication cloud provider.

With a lot of players offering access to “This overlap means there are
Edge networks at different layers of the redundant solutions for the market.
network stack, there are bound to be Enterprises do not want to rely on only
overlaps, Ivanov concedes, but actually one fabric, and the bigger they are the

"If you control the data journey, you control the

gateway, and you can control the gateway only if
you are directly involved in infrastructure"

Next-Gen
Automation Supplement || 27
Networks Supplement 27
 Next-Gen Networks Supplement >>CONTENTS

Building light
into chips Peter Judge
Executive Editor

The AI revolution needs so much

processing power, could it be time to bring
in a new generation of chips that combine
photonics with conventional silicon?

I
t’s well known that Moore’s Law
is coming to an end. We can no THE PROBLEM THE SOLUTION
longer expect processor power to
double every two years, as more
transistors are packed onto each
silicon chip.

That’s inconvenient for conventional

IT, which has been riding high on the
continuing dividend from Moore’s Law.
It’s potentially a disaster for artificial
intelligence (AI), which is on the verge of
a big expansion… but an expansion that
depends very much on fast processing.

One startup believes the answer is

to combine conventional silicon with
photonic processors that operate with
light. take as much as they can get, they'll spend ten years, the complexity of AI models has
any amount of money. Google, Microsoft, followed a 3.6 month doubling period.”
Amazon, and Facebook, will pay anything
The AI explosion The trouble is, even cheap general-
for these things.”
purpose silicon can’t keep up with that.
Artificial intelligence is in a furious
This is a recent development. After And, while it’s possible to throw extra time
growth phase right now, says Nick Harris,
surges in the 1960s and 1980s, AI research and resources at an AI in the lab, it needs
CEO of LightMatter: “People have found
was progressing slowly. Then in 2012, a fast performance when it is deployed in
use cases that are insatiable. They will
neural network called AlexNet, created real applications.
by Alex Krizhevsky, won an image “The challenge with AI is, you can
recognition contest running on low-cost train very big models, but if you'd like to
GPU hardware. deploy them and have people interact
That showed commercial possibilities, with them, the time between a user
Google bought Krizhevsky’s company, making a query and getting a result back,
is very important,” says Harris. “You need
and investment began.
real-time feedback. The big challenge in
“There was this massive investment the field is to build machines that can run
in scaling these things out,” says Harris. these huge neural networks so you get an
The investment bore fruit. “In the past answer back within milliseconds.”

28 DCD Supplement • datacenterdynamics.com

>>CONTENTS Light matters 

resistance, and don’t create heat - and

photons also travel faster than anything
else.

For some years, advanced computer

designs have tried to introduce
photonics, and use “electrons for
processing, photons for communication,”
in the words of John Sontag, an
HPE scientist (HPE is an investor in
Lightmatter).

Long-distance communications
use fiber optics, and those fibers now
penetrate deep into the racks of data
centers. “You have companies selling
100 Gig pluggable optics, and they're just
now deploying 400 Gig pluggable optics.
They send 400 gigabits per second of
data over the optical fiber to lace together
Silicon can’t keep up
racks and things that are spatially
Processor performance has been doubling separated.” “It’s just too early to have a co-
every two years for decades, since 1965, packaged optics solution that is ready for
Recent developments have allowed mass deployment and volume production
when Intel’s Gordon Moore noted the
trend. transistors and photonics to merge within the next few years,” said Dell’Oro
on the same wafer, in so-called “co- Group analyst Sameh Boujelbene in a
That’s been good, but that rate of packaged optics.” Initially, this has been comment to SDxCentral this year.
progress was not enough to keep up with seen as a way to reduce the size and
AIs emerging this century, says Harris:
“Even if you have the best case scaling for
electronics, you're not really powering
this.”

And to make matters worse, just at the

moment smarter AIs arrived, the rate of
silicon acceleration slowed.

Moore’s Law held because chip makers

could double the number of transistors
packed on a fragment of silicon every
two years, Now, while processors are
still packing more transistors, they are
running hotter.

“The reason we have this heat problem

is Dennard Scaling,” explains Harris. power consumption of those optical
Robert Dennard invented DRAM, and plugs, bringing the signals into the
observed that smaller transistors used less chip as light, instead of converting light
energy, scaling with their area: “Around signals to electrical ones at the borders of
2005, that broke down.” the CMOS chip.
Today’s fast processors use 300W and According to the roadmap, “optical
upwards, and Harris says that’s heading
components get closer and closer to the
for 1kW chips.
silicon until, eventually, the optics are
“We're still getting more transistors per 3D stacked and co-packaged with the
unit area. But you can't really use them, processors and networking chips, giving
because the cooling solution does not you very high data rates at low energy
support you using them. The chip will consumption.”
burn. You need to be able to develop chips
Intel has been demonstrating co-
that perform more operations per Watt.”
packaged optics for a year or more,
Broadcom has demonstrated a co-
Enter photonics packaged optics switch, and Marvell
What makes chips hot is resistance. bought photonics company Inphi for
Electrical signals face resistance as $10 billion in 2021, but the industry
electrons flow in current. By contrast, is skeptical about it coming into play
light signals don’t face the same quickly.

Next-Gen
Automation Supplement || 29
Networks Supplement 29
 Next-Gen Networks Supplement >>CONTENTS

Lightmatter’s optical network sends

signals with photons and runs cooler.
“For Passage, we're looking at the communication It’s also somewhat smaller, but is still
“inches across,” with Passage fitting into
between chips, and for Envise, the optical an eight-inch by eight-inch chip socket:
processing core helps with communication energy" “The biggest chip socket I've ever seen in
my life.”

Co-packaged optics could be useful commercializing it since 2018, with the It does, however, offer that “absurd”
for making the highly-interconnected aid of $11 million in startup funding. bandwidth: 768Tbps.
GPU systems used in training AI, but Wafer-size chips might sound like a
that still requires compute clusters with Going to silicon liability, given that all silicon wafers can
a “rat’s nest” of interlaced optical fibers, suffer from small point defects, so a large
comments Harris. “They're planning to The company has two products. Passage
wafer has a higher chance of failing. “We
lace together the processors inside the is an interconnect which takes arrays of
do a lot of work on yield engineering,”
server using the optics. When every chip traditional processors and links them up,
says Harris. “But there are not a lot of
is connected to every other chip using a using a programmable on-chip optical
transistors on the chip.”
fiber, there are performance benefits, but network.
it's very hard to service those things.” With few transistors, there’s less
“Lasers are integrated into the
chance of point defects: “We have very
Lightmatter’s approach is to push the platform, along with modulators and
low densities, so there's a very low
optical elements further inside the chip, transistors,” he says. “If you take a
probability of getting a point defect in
so all those interconnections are handled scanning electron microscope to the manufacturing that kills the transistor.
by a switchable photonic network within thing, you can see the waveguides - they The yields end up being high because it's
the silicon, that generates no heat, and are spaced about two microns apart, and not a very densely integrated transistor
takes up minuscule volume. are a few 100 nanometers wide.” circuit.”

“Fiber is macroscopic, it's on the order The other product is Envise, a general
of a millimeter,” he says. “Our devices are purpose cloud inference accelerator, Applications
two microns.” which combines computation elements,
with a photonic computing core. The first applications for this will be
This could drastically reduce the companies that do analysis of real time
hardware required, effectively integrating The promise here is to address the videos, says Harris. These could include
a complex AI training system onto a issue of AI processing speed: “We have security firms, but also companies
single chip: “If you open our server, about a 42x improvement in latency, monitoring a manufacturing line using
there's one chip in there. It contains all of because the processing is happening cameras to spot when a part has a defect.
the processors for the server. And they're at the speed of light. You're doing
It’s also potentially useful for speech
optically interconnected inside of the multiplication and addition as light flies
analysis and other AI applications: “It's
chip. And they can communicate with through the chip.
across the board.”
other platforms over optics as well.” The technology is still at an early stage,
There’s one common factor -
He continues: “Ultimately, what but Harris says Lightmatter has “about five
customers are interested in the
this thing does is extreme integration, customers,” who are large enterprises. The
“transformer” type neural networks
enabling everything with optical company has silicon in the lab, and will
pioneered by Google, and want to
have the chips on general availability later
interconnect, and allowing for really implement them more cheaply
in 2022.
absurd bandwidths.”
“The first application would be
“In the Passage case, we're looking at
And it’s done in standard processes principally trying to address dollars-per-
the communication between chips, and
available from merchant silicon fabs: “We inference cost. If you're a product person
in the Envise side, the optical processing
built our wafers with GlobalFoundries,” who is working on Google Cloud, there
core helps with communication energy,
says Harris. “We have transistors that are a lot of AI models you'd like to deploy,
and also offloads computer operations,”
are very close next-door neighbors, but you can't afford to, because the dollars
says Harris.
within 100 nanometers of the photonic per inference doesn't make sense.”
components. It's all monolithic.” The products are “big chips,” says
Will it all work? One positive sign is
Harris. Much like another AI chip startup,
The same etching tools make the the caliber of the engineers joining the
Cerebras, Lightmatter has found that
CMOS, and the photonic connections, company.
integrating multiple cores and a network
which are on the same nanometer scale as
can be done across a single wafer. Richard Ho, one of the leaders of
transistors, he says.
Google's custom AI chip family, the
Cerebras is further advanced
“We use all the same etching tools. So Tensor Processing Unit (TPU), joined
commercially, with products adopted Lightmatter in August, following Intel's
it's all completely standard CMOS. We
at the EPCC supercomputing center VP of engineering, data center, AI group,
use a ‘silicon on insulator wafer,’ which is
of the University of Edinburgh, and at Ritesh Jain. In May it hired Apple finance
used in the production of many electronic
Biopharmaceutical company AbbVie, director Jessie Zhang as VP of finance.
chips.”
among others. However, it has had to create
Harris and colleagues developed its own liquid cooling system to deal with The prospects for photonic computing
the idea at MIT, and have been the heat generated in the on-chip network. could be bright. 

30 DCD Supplement • datacenterdynamics.com

>>CONTENTS Time, revisited 

A second look
Sebastian Moss

Hyperscalers want to Editor-in-Chief

end the leap second,

as things start to turn
negative

M
eddling with time can have
unintended consequences.
In the data center world,
it only takes a second to
cause an outage or even
corrupt data.

But since 1972 we have done just that 27

times, adding a leap second every few years
in an effort to equalize two different ways of
tracking what the time it is.

Now, many in the industry are calling

for a rethink of how we approach time, and
demand the end of leap seconds.

Our species has always been in search

of precision, with complex mechanical
watches replacing centuries of hourglasses,
astrolabes, water clocks, and sundials.

But it was in the last century that we

made a breakthrough in precision: The
atomic clock, which measures time by
monitoring the resonant frequency of
atoms.

Hundreds of these clocks around the

world were used to create International
Atomic Time (TAI), a weighted average
of time as a constant duration defined by
cesium radiation.

But scientists soon realized that

something was amiss - the time did not 'Physics Laboratory: Time and Frequency Division.' By National Institute of Standards
match up to that of Universal Time, or UT1, and Technology
which is based on observed solar time,

Next-Gen
Automation Supplement || 31
Networks Supplement 31
 Next-Gen Networks Supplement >>CONTENTS

'Strontium Clock.' By National

Institute of Standards and
Technology

where noon is when the Sun is at its apex. concept of ‘now.’ which of course blows up everything in your
code," Obleukhov explained.
The Earth is not a perfect sphere, So enterprises turn to time servers
nor is its orbit a perfect circular ellipse. to tell their systems the time. These use "There are outages all across the industry
Complicating matters further is the fact Network Time Protocol (NTP) - a networking all around the world when leap seconds hit,
that the Earth's rotation has been slowing protocol for clock synchronization between where CPUs spin at 100 percent because of
due to tidal deceleration and other factors, computer systems over packet-switched, such events, where the only remediation
changing the length of a day. variable-latency data networks - to get was to go and physically reboot devices. This
within a few milliseconds of UTC. has happened again and again every leap
In the latter half of the last century, this
second."
change meant that UT1 was 1.3 ms slower A big virtual cluster of timeservers is
than TAI every day, on average. known as a pool, where a large number Each one has caused problems, taking
of computers volunteer to provide highly down platforms like Reddit, Cloudflare,
In 1972, the international reference time accurate time via NTP based on their own Foursquare, LinkedIn, and Yelp, among
scale Coordinated Universal Time (UTC) was source of time from a DCF77 receiver, others.
launched in an effort to combine these two WWVB receiver, or a GPS receiver, among
competing visions of time. others. "Throughout my career, I went through
multiple leap seconds, and everywhere it
It began based on TAI (with an initial Both Meta and Google offer their own was a disaster, and everything was falling
difference of 10 seconds), but periodically NTP service, based on their own atomic apart every time," Obleukhov said.
has whole seconds added to bring it closer clocks. "Every pool defines its own rules,
to the slower time tracked by UT1. Outside engineers have strong opinions," Oleg A report by the National Institute of
of the leap second adjustments, UTC is Obleukhov, the creator of Facebook's Public Standards and Technology (NIST) and
mapped to atomic time by a constant offset. NTP and cofounder of the company’s France's Bureau International des Poids
internal time card, said. et Mesures (BIPM) found that “contrary to
This is the time that the networked
our expectations, the number of problems
world relies on. Computers need to know It’s all a careful balance, where if one
reported has increased with time."
the precise time to communicate with each mistake happens, everything can come
other, with accurate time stamps required crashing down. In an effort to mitigate such a risk of a
for billing systems, database sorting, sudden change in seconds, Meta has begun
That's why periodically adding a new
network diagnostics, transactions, and 'smearing,' a concept first proposed by
second can be a significant threat to uptime.
more. If they get the time wrong, things can Google in 2011, instead of ‘stepping’ a whole
crash. When data centers receive the satellite second in one go.
signal announcing the leap second, they
Computers come with their own clocks, Smearing adds a couple of milliseconds
either show the impossible time of 23:59:60,
of course. But quartz oscillators drift, slowly every now and then over a longer period of
or they miss a second.
going out of sync with time, causing havoc time, reaching a full second just as the new
when multiple systems have a different That could cause "a negative number, leap second comes in.

32 DCD Supplement • datacenterdynamics.com

>>CONTENTS Time, revisited 

There are numerous ways to smear,

either by adding equal amounts of
milliseconds, or doing so in different "Defining a time scale that satisfies the needs of
amounts at varying intervals.
time and frequency users and is also
Google does it over 24 hours, while Meta
goes for 17. Alibaba is believed to smear for in agreement with astronomical phenomena
12 hours on either side of the leap second. is not straightforward and a series of trade-offs
"There are many different techniques for
smearing - all of them are bad," Obleukhov
are necessary"
said.

"You have to do it over many machines,

and this introduces errors between but that introduces its own risks, most NIST and BIPM aren't quite as aggressive,
machines,” he explained. notably that the networked world has never just yet. Researchers at the time institutes
tried this. suggested that perhaps a temporary answer
“Depending on how sensitive your lies in increasing the maximum difference
systems are you might have a problem. And "These events have never happened, so
between UT1 and UTC. That could mean a
when you’re smearing, if appliances get that it is almost a certainty that there will be
leap minute, or even a leap hour.
rebooted or if something else goes wrong, widespread errors in realizing the event, if it
then the chances of a fatal issue raises happens," the NIST and BIPM report states. The benefit would be that it would occur
drastically.” much less regularly, making risky events less
As it stands, if the current rate of change common. But there's a danger to that, they
Smearing is the best option we have at between UTC and UT1 continues, then admitted.
the moment, "but you still may get negative a negative leap second is expected to be
required by 2030. By making it a once-in-a-generation or
time," Obleukhov said.
more event, whole systems would be birthed
Meanwhile, public pools like NTP.org Given all this, Meta - along with Google, and die between leap events. Knowledge and
do not smear. “What you will end up doing Microsoft, and Amazon - suggest killing off preparation could be lacking, making the
if you join them is stepping, which is just the leap second entirely. They are joined in time change all the more dangerous.
dangerous,” he added. this recommendation by the NIST and BIPM,
although the time-tracking bodies have a "Therefore, it will be necessary to place
This is not the only problem. After slightly different approach. an increased emphasis on education and
decades of leap seconds being added as awareness ahead of such a step," they said.
the Earth's spin slowed, the planet began to There are still those that wish to keep
the status quo, arguing that scientists and Such a move would add a huge risk, but
accelerate in 2016, reaching its fastest spin
astronomers observing celestial bodies rely reduce the times we will face that gamble.
since the change in August.
"We do not consider that there is a 'perfect'
on UTC. Were it to move out of sync with
Why this is happening is not clear, but solution to the problem," the national bodies
UT1, then legacy equipment would need to
scientists have several theories. said.
be adjusted, and there could be a period of
Seismic activity such as the 2011 inaccurate astronomical observations and "Defining a time scale that satisfies the
earthquake in Japan shifted the planet's axis celestial measurements as a UTC-based needs of time and frequency users and
by 6.7 inches, which sped up the rotation. infrastructure has to be painstakingly shifted is also in agreement with astronomical
to UT1. phenomena is not straightforward and
Another potential reason is known as the a series of trade-offs are necessary. We
'Chandler Wobble,' where the movement of But Ahmad Byagowi, time appliance
consider that enlarging the tolerance
the geographic north and south poles causes project lead at the Open Compute Project
[between UTC and UT1] is a wise provisional
the planet to wobble, slowing it down - but and research scientist at Meta, argues that
solution, which should be re-considered
in recent years it has wobbled less. ultimately they will benefit from such a
when new discoveries and deeper
move.
Finally is our own impact on the planet. understanding could result in a better
Mountain ice caps have historically melted At the moment UTC and UT1 are already solution."
and refrozen, impacting our rotation like the out of sync, he reasoned, as their times are It is not clear whether either of these calls
arms of a spinning figure skater - when the only normalized when a whole second is will lead to immediate action. The quest to
arms are out, they move slower, when they added. Between those leap seconds "you kill the leap second faces an obstacle almost
are in, they move faster. have an error," he said. as inevitable as the passage of time: Inertia.
Now, due to anthropogenic climate "To those scientists that want to observe Time has no single ruler. The decision
change, that great mass of ice has melted the sky, we're suggesting that they will over the leap second will have to be agreed
and is not returning, instead staying at a always be able to go to a website that says on by multiple governmental, research,
lower altitude. 'this is the offset between UTC and UT1.' and non-governmental bodies, with its
It's much more granular, you can go into detractors having to navigate complex
Whatever the cause, we now face the first
milliseconds, and you can actually see politics and a natural unwillingness to
time our rotation has sped up since UTC
things much much better. That's what we're change.
began, potentially leading to a completely
proposing."
new challenge: The negative leap second. Such an effort kicks off this month, with
They want December 31, 2016, to mark a vote on the future of the leap second. The
Instead of adding an extra second, UTC
the date of the last leap second, with decision at the Consultative Committee
could remove one.
computers no longer having to worry about for Time and Frequency could help decide
This could also theoretically be smeared, interruptions to constant time. when the next major outage hits. 

Next-Gen
Automation Supplement || 33
Networks Supplement 33
IT professionals use data-driven,

insights to reduce downtime,

with EcoStruxure™ IT.

Gain the flexibility you need to optimize

uptime at the edge.
• Gain visibility and actionable insights into the health
of your IT sites to assure continuity of your operations.
EcoStruxure
• Instead of reacting to IT issues, take advantage of analytics IT Expert
and data-driven recommendations for proactive management.
• Keep costs under control by managing your IT sites more
efficiently. Choose to outsource to Schneider Electric’s
experts, address issues yourself, or leverage a mix of
both approaches.

ecostruxureit.com

©2022 Schneider Electric. All Rights Reserved. Schneider Electric | Life Is On and EcoStruxure are trademarks
and the property of Schneider Electric SE, its subsidiaries, and affiliated companies. 998-22321004_GMA
>>CONTENTS The cable ship
Thatcapacity
sinking crunch
feeling 

The cable ship

capacity crunch Dan Swinhoe
News Editor

Demand for cable

continues to increase, but
the fleet laying them is
small and aging

T
oday there are more than Report, after a splurge of investment around deployment or maintenance. Business is
400 or so subsea cables in the turn of the century, there were no new- booming.
operation, with dozens more build cable ships delivered between 2004
“Our fleet is now occupied up to 2024,”
due to enter service over and 2010, and only five ships were delivered
the next few years. These explains Jérémie Maillet, VP of Marine
between 2011 and 2020.
cables are the lifeblood of the Operations at ASN. “The contracts we are
And new ships aren’t being added at the negotiating right now are for installation
Internet; with the majority of the world’s data
same rate older ships are being retired. Only post-2024.”
flowing through fiber sitting on or under the
eight of those 60 ships are younger than 18,
ocean floor. Demand is so high, that cable companies
with most between 20 and 30 years old. 19
are over 30 years old, and one is over 50; the are often buying capacity or chartering other
However, the world’s supply of ships that cable ships to try and keep up with business.
can actually lay and maintain these cables is Finnish Telepaatti, built in 1978.
surprisingly small: just 60 ships worldwide. “There were a lot [of ships] built about 20 “Three years ago, we were not hiring
Most of those ships are long in the tooth; to 22 years ago,” says Gavin Tully, Managing external vessels apart from in specific areas
following a glut of new ships deployed Partner at Pioneer Consulting, which where local resources were mandatory
around the millennium at the height of the provides services on deploying submarine due to local regulation or customer
dot com boom, new ships have been few cable networks. “There's definitely a crunch requirements. At times [recently] we have up
and far between since. in the industry; projects are really at the to four external vessels working in parallel
mercy of ship availabilities.” with our own vessels on projects.”
As the industry sees huge demand for
new cables, largely driven by OTTs and “You can't just walk in and purchase At the same time, they are trying to
hyperscalers, there is an increasingly acute ship time,” he adds. “Scheduling is really keep an aging fleet out at sea for as long as
capacity crunch of available ships, meaning paramount right now; it takes time to get a possible instead of having them return to
projects are facing lengthy delays. slot in the ship schedules, and things are not port by transporting the cable to the cable
very flexible.” ships via freighters.

As an example, cable ship operator “With such a demand for installation

Cable ships: in demand yet Alcatel-Lucent Submarine Networks (ASN) activity, we can't really expect the cable ship
full of veterans ships rarely currently has a fleet of seven cable ships – a to come back all the way on a long transit
mix of purpose-built and retrofits, including back to the cable factory to collect cables,”
replaced
one in development that was originally a adds Mick McGovern, ASN’s Director of
According to the ISCPC, there are around cable ship that was moved to the Oil & Gas Projects. “We’re using freighters a lot more to
60 cable ships in the world. According to sector and is being re-fitted back to telecoms keep feeding the installation vessels cables in
SubTel Forum’s 2021/2022 Annual Industry use – with several dedicated to either cable the region that they’re working in.”

Issue 46 • November 2022 | 35

DCD Magazine #46 >>CONTENTS

Delays and re-routes cable back when permit becomes available smaller parts of a project. For example,
and going back and laying it.” funding a marine survey, then purchasing
While the existing fleet of cable ships had the cable, then permitting, etc.
been more than enough to keep up with “It can lead to you installing a system
industry demand since the dot-com days, during the period of the year that is “It's being bitten off in chunks, which
the recent boom in new subsea cable inefficient in terms of weather and create when you're asking someone to take a risk of
projects has seen the cable ship industry restrictions in terms of your ability to deploy giving me $5- $10 million versus $200-300
quickly become a seller’s market, where the and recover plows or land shore ends, etc.” million, clearly the risk tolerance is very
power is in the hands of the ship operators. different. ”
While hyperscalers might see a project
“The suppliers are in a good position slightly delayed, Pioneer’s Tully notes that Tully adds that the company is seeing
right now where they're basically able to say, the smaller cable projects are the ones that more phased implementations in longer
‘give me money and I'll give you a schedule. are more likely to be affected. projects. Instead of doing all 10,000
And if you don't have the money, come talk “The suppliers right now won't kilometers of a trans-Pacific cable at once,
to me when you do and I'll tell you what the commit to anything until you give them a for example, the project may be broken into
schedule is then,’” says Tully. “And that's a downpayment, and that downpayment is smaller chunks – i.e Asia to Guam first, then
very different situation than five years ago, also hand-in-hand with proof of full funding Guam to North America later – which lowers
where the suppliers would be elbowing each for the entire project,” he says. the financing hurdle and means a smaller
other out of the way for business.” window of ship time is needed at any one
“The suppliers are prioritizing the moment.
As a result, most projects will likely be hyperscalers, which is sometimes a
faced with delays. Even the hyperscalers, disadvantage to the smaller, more The most extreme public example of
which may get more lenient treatment entrepreneurial customers who may need a the impact of the cable ship shortage is in
as suppliers know they are good for the signed contract with the supplier so that due Canada. In May 2021, Maple Leaf Fibre, a
money and likely repeat customers planning diligence by the financier can be completed. Canadian project to lay fiber cables between

multiple projects, are seeing delays creep up. But the schedule inside that contract won’t Kingston, Toronto, and Montréal, scrapped
be confirmed until downpayment and proof plans to lay a cable under Lake Ontario due
“There’s definitely frustration on the
of full funding is made.” to a shortage of cable ships.
part of the developers, and I would include
the hyperscalers,” says Tully. “Projects are Even if a developer does have a schedule Announced in 2018 as a joint venture
taking longer than anyone is planning for. with a supplier, if there are delays achieving between Metro Optic and Crosslake Fibre
The majority of the projects that we see financial close, the schedule can easily slip along with Utilities Kingston, the cable
finish later than when they were initially by six months or more, or see price increases was set to be terrestrial between Kingston,
conceived.” to hold the schedule. Ontario, and Montréal, and under Lake
Ontario westwards from Kingston to
Projects that might include a year buffer “On existing projects where all the money
Toronto. However, a shortage of cable-
to allow for financing, permitting, and other is committed, we're still seeing delays creep
laying vessels led to a change of plans,
delays, are “plowing through” those time up, and it's creating a lot of tension between
with the whole cable system now due to be
contingencies by anything from six months clients and suppliers. Some of these delays
terrestrial, running from Toronto east via
to more than a year later than was initially are in the order of six months, even when
Kingston to Montréal.
planned. a project is already fully financed, as the
suppliers themselves are encountering a Fergus Innes, chief commercial officer
Delays caused by external factors can also
lot of difficulties scheduling all the different of Toronto-based Crosslake, told Capacity:
have a knock-on effect. ASN’s McGovern
projects and prioritizing things.” “Vessel availability [is] one of the reasons
says. “[Delays with permits] lead to huge
we have pivoted from a subsea design to a
inefficiencies in the operations of the As a result, smaller cable companies are
full terrestrial build on our Maple Leaf Fibre
vessels. You might have the cable loaded and beginning to look at taking a ‘disaggregated
project.”
the configuration on board, but if a permit approach’ to financing a cable project;
is not in place or it's delayed and unclear instead of a turnkey project and a small The requirement of needing to take a
when it is going to be freed, then you end number of large financiers, companies are ship that was big enough to carry and lay
up turning over the cable, laying the system working with smaller, more nimble and risk- the cable yet also small enough to fit along
in a different direction and then turning the tolerant financiers who are willing to finance the St. Lawrence River lock system likely

36 | DCD Magazine • datacenterdynamics.com

>>CONTENTS That sinking feeling 

added further complexities. DCD reached

out to Crosslake, who declined to comment
further. “The suppliers are prioritizing the hyperscalers,
The Maple Leaf route was unique which is sometimes a disadvantage to the
in having a terrestrial route as a viable
alternative – with the ‘wet’ route likely smaller, more entrepreneurial customers"
a cheaper option in the original project
However, Pioneer’s Tully notes that ship These ships can be in service for 20
timelines – with most other cables unlikely
owners and operators are incentivized to years or more, and it seems many cable ship
or unable to make such route adjustments
maximize every available minute of these owners and operators are reluctant to pile
and instead will have to accept delays.
ships to be working, due to the high cost money into the next generation of vessels
of standby, which can run into the tens of when many automation and sustainability
thousands of dollars per day. technologies are still immature. However,
Where’s the next-generation Orange Marine has said the replacement for
“The worst possible thing you can do
fleet? the Raymond Croze will include electrical
is have these ships sitting around idle. But
storage back-up using batteries when she
The main issue causing the crunch is the now they have a book of business and can
launches in 2023.
lack of ships, but the fact the fleet is largely say this ship is now booked for the next 24-
full of vessels closer to retirement than plus months, and that is exactly what these “The question of the renewal of the
launch isn’t helping. companies want.” worldwide fleet is still a big question mark.
We don't have the financial capacity of a
“We're seeing projects now that have New ships can cost upwards of $100-150
Maersk or a big company owning tens or
been delayed because of just maintenance million, and delivery can take several years.
hundreds of vessels,” says ASN’s Maillet. “But
issues with ships,” says Tully. “That’s not due The soaring price of steel also means the
at some stage, we will have to face and take
to negligence; these ships are just old and costs for new ships are going up rapidly.

there's no slack built into the schedule.” Many cable ship owners and operators the decision to start to renew the fleet.
are reluctant to make such a large-scale
The equipment and technology aboard “The market ramped up very quickly
investment in new builds as the costs and
the vessels are always improving. And new during the last few years. Nobody really
business case can be hard to justify, and
ships do occasionally come into service, but anticipated it, and the lead time for new
there’s no guarantee that demand will
they are often on a one-in, one-out basis as build construction is a minimum of three
continue at the current red-hot levels once
older vessels are retired. And often even then years. Will this market will be sustainable
any new ships do enter service.
the recent trend has been to retrofit older for the next 10 years? Maybe it's not enough
ships to save costs and speed delivery. At the same time, change is on the to defend a strong return on investment to
horizon. The future of shipping involves build new cable ships.”
In 2020, Orange subsidiary Orange more automation – partly to reduce costs
Marine said it would build a new cable ship In the meantime, retrofits of smaller ships
and partly to deal with an ongoing skills
designed to help maintain both fiber and are becoming more common for smaller
shortage – yet remote navigation and
power cables, due for launch in 2023 to projects. Maillet notes more retrofits and
maintenance technologies are still in their
smaller, more specialized vessels tailored to
replace the 40-year-old CS Raymond Croze. nascent stage.
specific roles rather than very large multi-
Its last new ship was the Pierre de Fermat in
The fossil-fuel-reliant industry is purpose ships may be a way forward in the
2014.
looking to decarbonize over the next short term to alleviate some of the capacity
This year saw SBSS launch a new cable couple of decades and meet the various crunch in a more cost-effective way.
ship, CS Fu Tai. Built in Spain in 2007 as 2050 net zero goals being set by countries “In the future, we may design different
an offshore construction vessel, SBSS the world over. However, large-scale types of vessels, more specialized and not
purchased the Fu Tai in 2021 and converted batteries that can reliably support large capable to do everything,” he says, “but
her to a bespoke vessel. South Africa’s vessels at sea for long periods are still in extremely efficient for what they have to
Mertech Marine recently announced the development, and the supply chain to provide in terms of service.”
retirement of cable retrieval ship MV Lida. It ensure such technologies can be supported
plans to replace the vessel, but hasn’t made wherever in the world a cable ship might be For the longer trans-oceanic projects,
any announcements yet. needed are still a ways off. however, the cost and wait times for a new

Issue 46 • November 2022 | 37

DCD Magazine #46 >>CONTENTS

The future: More crunch, more

long-term charters?
There is no magic bullet to solve the capacity
crunch; cable demand continues to go up,
and the number of cable ships to deploy
and maintain them is unlikely to increase
substantially in the near future.

As a result, it remains a seller’s market

and cable owners should likely expect longer
timelines on their projects.

Though it would likely be affordable to

companies making billions of dollars of
profit per quarter, none of the people DCD
spoke to for this piece predict it is likely that
a hyperscaler such as Google will build or
buy a cable ship or cable ship company in
the short to near term.

While most admit it could be remotely

possible in the long term, a more likely
scenario would be to see a hyperscaler lease
a boat on a long term charter.

This is a common practice in the Oil &

Gas industry, and also happens in the telco/
subsea fiber sector, and would allow them
to benefit from having ready access to such
a vessel and skills without the long-term
cable ship capable of carrying thousands of While it hasn’t made an official investment or ongoing management.
miles of cable means new ships are unlikely announcement, SubCom, a major player in NEC recently signed a long-term charter
for now. the subsea cable space, is reportedly looking contract with Global Marine, securing the
to exit the maintenance market and focus its Normand Clipper for approximately four
ships entirely on cable laying, meaning there years from September 2022 to May 2026.
Maintenance: thin margins will be even fewer cable ships dedicated to NEC said the contract “strengthens its
mean even less chance of new maintenance at a time when there are ever provision of submarine cable systems” and
ships more cables entering operation. allows it to “respond to expanding demand
While the lack of cable ships is affecting the for submarine cables.”
On the plus side, maintaining cables
deployment of new cables, a similar issue is often less demanding than laying them Built in 2001, the 127-meter ship can
may be bubbling in the background for the and often doesn’t require as much cable or carry up to 5,000 tons of cable, equating to
maintenance of existing cables. heavy equipment such as plows. This allows around 7,000km worth of fiber.
While Global Marine’s maintenance for smaller vessels that may not need to go
“Until now, NEC has procured submarine
account director Steve Holden tells DCD out to deep sea, as most cable breaks are
cable-laying ships for each project
there is probably currently ‘sufficient repair close to shore, meaning there is a greater
separately,” the company said. “In order to
tonnage’ for the current market, the market opportunity for customizing older ships to
respond to the growing demand for new
is facing similar barriers to investing in fill shorter-term gaps.
submarine cables due to the recent spread of
new vessels, driven by narrower margins 5G and the increase in data traffic between
ASN’s Maillet said he’d like to see the
compared to the cable deployment side. The data centers in various countries, NEC has
hyperscalers and other companies invest
company is looking to extend the lifespan chartered a long-term dedicated cable-
in larger, longer contracts that would allow
of its fleet – seven ships including those on laying ship for the first time.
cable ship companies to invest in new and
charter ranging from 10 to 30 years old – to
converted ships. But as demand for subsea cables continues
40 years.
“There is a hard competition on the to grow, something may have to break.
“The maintenance contracts are not
really long enough and not conducive to maintenance market, but the duration of the “The situation does seem unsustainable
any new build globally,” he says. “Many contract for the moment is probably not at in the long term. In the near and medium
just see it as a cost to be driven as low as the right level,” he says. “The cable system term, I think it'll just continue and we'll
possible. owners should understand that by keeping band-aid it as we go,” concludes Tully.
pressure on this market, it will prevent the “It doesn't seem sustainable to continue
“At the moment the economics for cable ship operators’ abilities to invest. without a big delivery of new ships. At some
replacing a traditional cable ship don't stack up
point, there has to be a tipping point.”
at all. If we saw that a conversion opportunity “In a sustainable industry, three or four
arose that was suitable, then we would take it. vessels should be under construction or “I wonder if it will mean the entrance
But we believe that it's actually better to extend conversion right now to address the demand of new companies stepping in and saying
the life of the fleet at the moment.” and to be able to cover all the repairs.” ‘we're going to build new ships." 

38 | DCD Magazine • datacenterdynamics.com

>>CONTENTS That sinking feeling 

A BRIEF HISTORY OF CABLE SHIPS

W
hile today cable ships are custom- attempt to lay a transatlantic cable in 1857 cable repair. She was scrapped in 1922.
built specifically to lay subsea fiber required two vessels, was plagued with One cable project was responsible for not
cables, the first ships involved in problems, and quickly failed once activated. only the first ever loss of a cable ship, but the
deploying undersea telegraph cables were Two converted warships, the HMS Agamemnon second also. Though details are sparse, the
paddle ships chartered and customized where and USS Niagara, borrowed from their ill-fated CS Gomos was reportedly rammed by
possible. respective governments, were loaded with another ship in the 1870s while laying a cable
One of the first offshore cable proof of cable; both ships were needed as neither could between Brazil and Uruguay for the Brazilian
concepts was conducted in 1849 by Charles hold 2,500 nautical miles of cable alone. Submarine Telegraph Company. Chartered
Vincent Walker of the South Eastern Railway At the first attempt, cable laying began off alongside CS Ambassador for the project,
Company: Walker successfully laid two miles Ballycarbery Castle in County Kerry, on the she was the first cable ship ever to be sunk.
(3.2 km) of cable in UK waters from the ship southwest coast of Ireland, and broke on the Replacement cable was manufactured and
Princess Clementine off the coast of Folkestone first day. It was grappled and repaired, but broke the CS La Plata chartered. However, La Plata
to the shore where it connected to the railway again over a region of the North Atlantic nearly foundered in the Bay of Biscay with the loss
telegraph lines, sending telegraph messages 3,200 m (10,500 ft) deep known as Telegraph of 58 lives. The Ambassador did eventually
from the ship to London. The Clementine Plateau, and the operation was abandoned for complete the laying.
was reportedly a 147-ton, 180-hp iron- the rest of the year. Around 300 miles of cable The most recent cable ship to be lost was
hulled paddle steamer launched in 1846 as were lost, but the remaining 1,800 miles were KT Submarine’s CS Responder. Built for Maersk
a passenger ferry across the English-French sufficient to complete the task. in 2000 and belonging to KT Submarine since
Channel that was briefly used as a transport A year later, after improving the mechanisms around 2016, she sank in September 2020 in
during the Crimean War in 1853. for rolling out cable, the Agamemnon and the East China sea off the coast of South Korea.
English cable pioneer John Watkins Brett's Niagara tried again. The vessels arrived at A fire broke out on deck while laying cable, and
Channel Submarine Telegraph Company was the middle of the Atlantic, spliced cable the ship sank due to the flooding caused by the
the first to lay a cable between England in from the two ships together and headed off; fire fighting. No one was hurt and the 60 crew
France. In 1950, the converted paddle tugboat Agamemnon east towards Valentia Island, and were evacuated to a nearby smaller cable laying
Goliath laid an unarmored cable between Niagara westward towards Newfoundland. The ship working in tandem with the Responder.
Dover and Cap Gris Nez in France. The cable cable broke three more times. A third attempt The first trans-Pacific telegraph cable from
failed the night after its first test, possibly due was successful, though the cable was damaged San Francisco in the US via Hawaii, Midway,
to damage by fishermen. Despite its status as within a few days after misuse by an engineer and Guam to Manila in the Philippines, and
the first cable ship, very little is known about and failed within a month. onto China and Japan, was laid around
the Goliath; though it was likely a wood paddle A second, more successful transatlantic 1901-2 by the India Rubber, Gutta Percha and
tug built in 1846 measuring around 100ft and cable was laid by the SS Great Eastern in 1866 Telegraph Works Company using CS Silvertown
100hp. and the ship, unlike its predecessors continued (previously the Hooper), and the Telegraph
A year later, a stronger second cable was to be used specifically for cable operations Construction and Maintenance Company
laid by the reconstituted Submarine Telegraph for years afterwards. An iron sail-powered, (Telcon) using CS Colonia and CS Anglia, two
Company from a government hulk, Blazer, paddle wheel, and screw-propelled steamship custom-built ships.
which was towed across the Channel. The designed by English engineer Isambard The first submarine transatlantic telephone
cable was laid between South Foreland and Kingdom Brunel, she was the largest ship ever cable system, TAT-1, was laid between Oban,
Sangatte with the Blazer under tow from two built at the time of her 1858 launch. Scotland, and Clarenville, Newfoundland in
tugs. Originally a passenger ship before being the 1950s by the cable ship HMTS Monarch, a
A month later the steam tug Red Rover was contracted out for cable laying in 1865, she was successor to the original Monarch and built in
tasked with replacing a temporary part of the converted to hold 22,450 kilometers (13,950 1946.
second cable with a new section of armored mi) of cable. After a successful laying project TAT-8, the first transatlantic fiber optic cable,
cable, but weather and navigation issues meant across the Atlantic, the Great Eastern continued landing in Tuckerton, New Jersey, Widemouth
it missed a planned rendezvous with HMS to lay and repair subsea telegraph cables until Bay, England, and Penmarch, France, was
Widgeon which had been tasked with making the 1880s. Later re-fitted as a liner, then a laid in 1988 by CS Long Lines (owned by
the splice at sea. The Widgeon did eventually showboat, and then used for advertising, she AT&T), CS Alert (BT), and CS Vercors (French
make the splice at a later date. was scrapped in 1890. Telecom). Capacity on the cable was reportedly
The paddle steamer Monarch, built in the UK The CS Hooper, built in 1873 in Newcastle, reached within eighteen months, despite some
in 1830, was the first ship to be permanently was the world's first purpose-built cable-laying predictions it would take a decade and other
fitted out as a cable ship and operated on a ship. It was designed to carry the whole of the suggesting it would never be filled and no other
full-time basis by a cable company, and was cable to be laid between England and Bermuda cables would be needed.
the first of a series of cable ships named in that for the Great Western Telegraph Company, Long Lines, built in 1961, was involved in a
regal fashion. however the project was abandoned. It laid number of cable firsts. The ship also laid the
The vessel was acquired and converted by a number of cables for the company before first trans-Pacific telephone cable, TRANSPAC-1
the Electric Telegraph Company in 1853 and it was sold to the India Rubber, Gutta Percha (TPC-1) in 1964; and laid TPC3, the first trans-
subsequently laid a number of telegraph cables and Telegraph Works in 1881 and renamed Pacific fiber cable along with CS KDD Maru.
around British and European waters. After Silvertown. A series of dedicated cable ships,
The ship was acquired along with CS Charles
nationalization in 1870, Monarch irreparably including the CS Faraday, followed shortly after
L. Brown by Tyco International in 1997 when it
broke down on her first cable mission for the the Hooper.
bought AT&T Submarine Systems (which was
General Post Office and was turned into a coal The CS H. C. Oersted, built for the Great spun out in 2000 and now known as Subcom).
hulk. Northern Telegraph Company in Denmark in As with all these ships, she wasn’t saved for
Though technically successful, the first 1872, was the first ship specifically designed for posterity and was sold for scrap in 2003. 

Issue 46 • November 2022 | 39

Data centre
battery
solutions
High energy Maintenance
in a compact free
design Our maintenance free products are easy, quick to
install and reduce overall operating costs during life.

Improved technology of inside components like

advanced alloys and manufacturing processes allow
the highest performances in a very compact design.

Designed
Safety first for life
All our products are designed and tested to
Flame retardant, high quality ABS plastic casings ensure the best performance for the whole life.
ensure maximum safety.
DCD Magazine #46 >>CONTENTS

Simulating the
flamingo universe, and Sebastian Moss
Editor-in-Chief

other challenges at
trillion-particle scales
At a university campus
S
ince man first looked to the skies, we have tried to comprehend the
cosmos.

in England, a RAM- But peering outwards is just one way to help understand
the universe. Another answer lies within, in the highly detailed
simulations now possible thanks to profound microprocessor
heavy supercomputer advances and decades of investment in high-performance computing (HPC).

tries to unlock cosmic In the north of England, one such supercomputer hopes to do its part
to present a history of the universe in unprecedented detail, providing new

secrets
insights into how we came to be.

Earlier this year, as the UK suffered an unprecedented heatwave caused by

climate change, DCD toured Durham University’s data center at the Institute
for Computational Cosmology (ICC), and learned about its most powerful
system, the Cosma-8.

Cosma-8 is part of the UK government’s Distributed Research

using Advanced Computing (DiRAC) program, which is formed of five
supercomputers around the country, each of which has a specific unique
feature.

In the case of the Durham system, that differentiating factor is its

Images by Sebastian Moss
breathtaking amount of random-access memory (RAM).

Images Credit: Sebastian Moss

42 | DCD Magazine • datacenterdynamics.com

>>CONTENTS Escape
Space
Ukraine
is big 

"For the full system, we have 360 nodes, RAM to their limits, fully connected by a understand the importance of dark matter in
46,000 cores, and very importantly for us PCIe-4 fabric. “Although our system isn't as the formation and evolution of the universe.
a terabyte of RAM per node - that's a lot of big as many of the larger systems, because
RAM," Dr. Alastair Basden, head of the Cosma "There's actually more dark matter
we have this higher RAM per node, we can
service, said. than normal things in the universe," the
actually do certain workloads better,” Dr.
assistant professor at UKRI FLF in Durham's
Two nodes in the system go even Basden said.
department of physics, said.
further, cramming in 4TB of RAM per node. One example is the MillenniumTNG-XXL
"These are for workloads that don't scale as simulation, which aims to encapsulate the “The normal matter - which is what
well across multiple nodes. So things like large-scale structure of the universe across galaxies are made out of, along with the
accessing large data sets and code which 10 billion light years. “It's basically the largest Solar System, planets, us, everything in the
aren’t very well parallelized," Dr. Basden said. simulation of its type that can be done universe that we can observe, basically -
anywhere in the world,” Dr. Basden said. includes only a small portion of the matter
This huge amount of RAM allows for
and energy in the universe.”
specific scientific problems to be addressed “So this is 10,2403 dark matter particles,
that would otherwise not be possible on this is trillion particle regimes - a large step Visible matter makes up just 0.5 percent of
conventional supercomputers. up from anything simulated previously,” the universe, with dark matter at 30.1 percent.
he said, “You can begin to see within the The final 69.4 percent is dark energy.
But more on that later, first a quick
rundown of the system's other specs: simulations it actually building spiral To understand how these forces interact
It boasts dual 280W AMD Epyc 7H12 galaxies and things like that, all from the requires enormous computing power.
processors per node with a 2.6GHz base physics that we put in.” “Earlier efforts only looked at dark matter
clock frequency and 64 cores, installed in The simulation takes data from distribution and ignored the more complex
Dell Cloud Service C-series chassis with a 2U telescopes, satellites, and the Dark Energy systems,” Dr. Fattahi said.
form factor. It also has six petabytes of Lustre Spectroscopic Instrument (DESI) to see “But we want to include more complex
storage, hosted across 10 servers that have “how well we can match what we get in our phenomena in the models that we're
their own two CPUs and 1TB of RAM. simulator to what is actually seen in the sky,” using,” she explained. “Now on Cosma-8,
The supercomputer uses direct-to-chip Dr. Basden explained. “That then tells us
we can basically run a full hydrodynamical
cooling, and a CoolIT CDU. more about dark matter.”
simulation, which means we include all the
The MillenniumTNG-XXL simulation complex procedures like gas pools, stars
You may notice a distinct lack of GPUs,
began in July last year, taking up a huge forming and exploding into a supernova, as
despite their usefulness in a number of other
amount of computing resources. “We well as supermassive black holes.”
simulation-based systems.
dropped about 60 million CPU hours on
One of the flagship projects on Cosma-8
"Basically the codes that we're doing don't that,” Dr. Basden said.
is the ‘Full-hydro Large-scale structure
match well to GPUs. There are efforts that are
“A large amount of memory per node is simulations with All-sky Mapping for
going on to port these codes to GPU, but the
absolutely essential. HPC codes don't always the Interpretation of Next Generation
uplift you can get in performance is a small
scale efficiently, so the more nodes you Observations’ study, or, as it is more
factor rather than large," Dr. Basden said.
use the more your scale goes down. Your commonly known, the FLAMINGO
However, the data center is home to a simulation would take longer and longer simulation.
two-node cluster funded as part of the UK's to run until you reach a point of no return.
“So FLAMINGO is at the cutting edge,”
Excalibur exascale efforts that have six AMD So it wouldn't have been possible without a
Dr. Fattahi said. “MillenniumTNG-XXL is
MI100 GPUs in it. "MI200 GPUs should follow machine designed specifically for this."
a slightly bigger volume, but doesn't have
shortly," the researcher added.
Dr. Azadeh Fattahi is one of the hydrodynamics. Compared to anything that
Cosma-8, however, has no plans for researchers trying to take advantage of has been done with hydrodynamics it is the
GPUs, instead aiming to push CPUs and the machine’s unique talents, seeking to biggest in the world.”

Issue 46 • November 2022 | 43

DCD Magazine #46 >>CONTENTS

he said. "Most of the time we use free air

cooling, but days like that we use an active
chiller. That means that most of the year we
have a PUE (power utilization effectiveness)
which is about 1.1, which is pretty good, and
then it can get up to around 1.4."

It has not always been easy, however,

he admitted. "For the last year or so, the
generator wasn't kicking in, so if the grid had
gone down we only had an hour on the UPS.
Fortunately, that didn't happen."

The generator is now fixed, but there still

exists another risk: "The chillers are not on
FLAMINGO’s simulated universe is article goes to print it is in a different turmoil UPS, so if the power dies the UPS will take
about 8 billion light years, featuring 5,0003 - but Dr. Basden is confident that it is on its over compute and the chillers will have to
elements of dark matter and 5,0003 of gas. way. be brought back to life by the generator,"
“This is the largest number of resolution he said.
Cosma-8 was funded under DiRAC II,
elements that have been run on a hybrid
with the scientific community building a "That doesn't always happen. Once I
simulation anywhere in the world,” she said.
case for III. "We put it to the government," Dr. was sitting in the data hall and the chillers
It took most of Cosma-8 working for 38
Basden said. "They said 'fantastic, but there's went down when we were testing stuff," he
straight days to finish.
no funding.' recalled. "I was like a frog in boiling water.
Dr. Fattahi's team uses these giant models I was just sitting there getting a bit warmer
"Year after year, we're waiting for this
to then zoom in to work at a comparatively and warmer. It got quite warm in here, and
money. Finally, at the end of 2020 they
smaller scale, operating at ‘simply’ the I was like ‘guys what's happening?’ A circuit
said ‘you can have some of the DiRAC III
galactic level. By choosing a smaller chunk breaker had tripped."
funding, but not all of it.’ We're still waiting
of space, she can focus the computational for the rest, hopefully it will come this year, Part of that lack of perfect redundancy
power while keeping the rest of the universe maybe next.” is down to requirements- unlike a
at a lower resolution.
When the money comes, and how much commercial provider that cannot go down,
“I study the low mass, the very small dark they get, will define phase two of the system. research supercomputers can be more
matter clumps, and dark matter halos,” she Lustre storage will likely be doubled, and it lenient with downtime (for example it has
said. “It turns out that small galaxies have will probably use AMD Milan processors. three multi-day maintenance outages a
a lot of dark matter, they are the most dark year), so universities are better off spending
matter dense dark galaxies in the universe. "Depending on timescales, we might get the money on more compute than more
The question that derives my research is some [AMD] Genoa CPUs, where we think redundancy.
what we can learn from the small-scale we could go up to 6TB of RAM per node," Dr.
structures about the nature of dark matter, Basden said. "And we have use cases for that." Another issue is the location: The
which is a fundamental question in physics.” data halls are within the Institute for
When that happens, the data center will
Computational Cosmology, a larger building
Even in these smaller simulations, the be set for a reshuffle. The data hall currently
built for students and researchers.
scale is still immense. Astronomers use holds Cosma-6, -7, and -8 (with -5 in an
solar mass as a unit of measurement, with adjacent room), but is at capacity. In the future, as they plan to move into
one solar mass equal to that of our sun. the next class of computing, the exascale,
"Cosmos-6 will be retired. Its hardware
“The target resolution is about 104 solar they will have to look elsewhere. "We are
dates from 2012 and it came to us
mass,” Dr. Fattahi said. “FLAMINGO has a going to need to build a new data center for
secondhand in 2016."
resolution of 108.” that,” he said. “I think we are looking at a 10-
Each system draws around 200kW for 15MW facility, which by the time we get to
Again, this simulation would not compute on a standard day, with around 10 exascale is achievable," he said.
have worked without the high RAM, Dr. percent more for the cooling demands.
Fattahi argued. “If we go over too many The only official exascale system
connections, the lines become quite slow, so "Sometimes there can be heavy workloads currently is the US' Frontier supercomputer,
we have to fit these simulations in as small a and it reaches about 900kW; our total feed which has a peak power consumption
to the room is 1MW. So we're getting close to of 40MW (but usually is closer to 20MW).
number of nodes as possible,” she said. “The
where we wouldn't want to have much more However, by the time the UK government
1TB per node allowed us to fit it into a couple
kit without retiring stuff. Yesterday we saw a will fund such a system, advancements will
of nodes, and then we could run many of
90 percent load." have brought that power load down.
them in parallel. That’s where the power of
Cosma-8 lies.” That day, the hottest on record in the
By then, we might also have a better
country, taxed data centers across the UK. It
The hope is to make it more powerful understanding of how the universe
brought down Google and Oracle facilities,
if and when more money comes in. The works, with scientists around the world
but the Cosma supercomputers ticked on
exact roadmap to that new funding is not now turning to the simulations built at
unperturbed, Dr. Basden said proudly.
known - when we visited the facility the Cosma-8 to help unpick the complexities
UK government was in turmoil, and as this "We survived the hottest temperature," of our cosmos. 

44 | DCD Magazine • datacenterdynamics.com

HVO.KOHLERPOWER.COM

FOR TODAY’S GENERATORS, FOR TOMORROW’S GENERATIONS.

WITH 90% LESS CARBON,

OUR JOURNEY TOWARD CARBON ZERO
BEGINS RIGHT HERE, RIGHT NOW.
How do we produce the energy and performance you’re used to
from a back-up generator, while producing 90% less carbon? Simple.
By powering it with Hydrotreated Vegetable Oil (HVO) a fossil-free and
100% renewable fuel. No generator modifications required. No waiting
for future engineering developments. Our journey to carbon zero begins
right here, right now. And with a KOHLER® generator, so can yours.
DCD Magazine #46 >>CONTENTS

Why telcos are

switching off legacy Paul Lipscombe
Telco Editor

networks, and what it

means for 5G
T
Network operators he discussion about the rollout and
impact of 5G has dominated the

are scrambling telco sector for a number of years,

as the industry looks to the future

to make the most

for new markets to conquer.

But we can’t forget about our past - before

out of their 5G 5G there was 4G; before this, there was

3G, 2G, and 1G. The launch of 2G in 1991
marked the beginning of digital mobile
networks, but why telecommunications, as 1G was analog, while
3G in 2001 was when it really hit its stride.

does this mean Decades later, those networks are still

going. Even 1G survived until 2017, when the
switching off 2G last network was reportedly switched off in
Russia.

and 3G networks? Now, as operators across the world focus on

upgrading their 4G and 5G networks, it’s come
to the point where these operators are finally
turning off 2G and 3G services.

And with good reason, with over half the

countries (53 percent) in the world forecasted
to have launched 5G services by next year,
according to analyst firm CCS Insight's latest
predictions for 2023 and Beyond.

Why are we retiring legacy

networks?
The refarming of spectrum is essential to
driving the full potential of what 5G can do and
ultimately bring, Wireless Logic group head of
MVNO Paul Bullock explained.

“Modern mobile core technology is not

focused on 2G or 3G, so as operators want
to upgrade their infrastructure, having to
maintain the services either imposes greater
costs on them or substantially limits what they
can do with their upgrades,” he said.

46 | DCD Magazine • datacenterdynamics.com

>>CONTENTS Escape
Moving
Ukraine
on 

Bullock also notes that 2G and 3G just

who will be impacted. The Alarm Industry He predicts that it will be hard to
aren’t required anymore, and it’s in the
Communications Committee found in compromise and make everyone happy
mobile operator’s interests to focus on
a survey of its members that about two when 2G services are eventually retired
pushing more modern technology. It’s just
million security, fire, and medical alert and could be expensive for some of
not profitable for MNOs to push these older these businesses to upgrade replacement
devices remained on 3G. Senior citizens
networks. technologies such as LTE-M, NB-IoT, LTE
who still have their old cell phones may be
“So, the operators are trying to find a impacted, too. CAT-1.
way to be more efficient in their spectrum “There's a lot of stuff in IoT that is still
As of mid-2019, there were 80 million
allocation, and also more modern in building active 3G devices in use across North attached to 2G networks. Not all those people
infrastructure which will enable them to be America. are going to be able to be made happy,
more profitable because they just deliver one and so there will be an impact on these
common plane for network access for all the 2G is equally used in rural areas and for businesses,” said Bullock.
consumers.” some early smart meters and Internet of
Things and M2M services that still rely on 2G “This flows into the replacement
Because 5G is effectively upgrading and for support. technologies of either LTE-M, NB IoT, or
enhancing the 4G networks, 4G will not be LTE CAT-1. And all of the IoT businesses that
shutting down anytime soon, with many 2G may cling on a little longer, but have a large 2G installed base have been
observers agreeing that 4G still has a lot to 3G’s shutdown is proceeding at pace. For frantically figuring out what to do next for a
give. example, Italy’s Telecom Italia (TIM), recently few years, and they're mostly solving these
shut down its 3G service, but has yet to problems now.”
Bullock agrees and says that 4G is still confirm when its 2G service will go offline.
evolving, and the shutdown of 3G services Bullock warns businesses that don’t
won’t have any immediate impact on these Bullock adds that it’s much more upgrade their IoT devices that it will be more
services. straightforward to switch off 3G services expensive in the long run, and says their
than 2G services, labeling 3G as a “stopgap technology will stop working, with little that
“4G is completely mature and is still technology.” can be done.
evolving. There'll be new releases of 4G
software stacks, that will fit into existing “Switching off 3G is more straightforward
infrastructure. Personally, I don't see any from a revenue and also a direct impact
material impacts on the state of 4G when point of view,” he said. “There’s just not that Leaders in the switch-off
you retire 2G and 3G.” many 3G things out there, it was almost a bit
The US has been one of the leading markets
of a stopgap technology.
for switching off these services, observed
“There are no [new] consumer devices Bullock, who expects Europe to catch up.
Refarming key to unleashing that are 3G-only, and there aren’t very many
AT&T and T-Mobile terminated 3G
5G IoT devices that are still around that use 3G,
services earlier this year, while Verizon plans
so it’s easier to kill off.”
Bullock does make a point to say that the to shut down its 3G network by the end of
current 5G we’re seeing now is more akin to As for 2G, he explains that it’s a bit more this year.
a faster version of 4G, with plenty still left in complex, with a lot of smart meters using 2G
For comparison in the UK, the
the tank to extract. in the past two and a half decades.
government has given all four MNOs by
Dropping the older technologies “will “For 2G, the biggest victims will be the 2033 to retire both 2G and 3G services,
mean that operators have network resources smart metering industry. There have been a although it’s widely anticipated that services
lot of smart meters around the world, in the will be phased out much sooner, Vodafone,
to deploy 5G from spectrum to people, to
last 20 to 25 years, and they'll have 2G SIMs for example, is planning to switch off its 3G
various technical capacities it takes to deliver
in them, and 2G modules. service next year.
cellular network services,” he said.
“So, it’s not just a matter of driving from While in Belgium the switch off of 3G
“This is not going to make 5G
one place to another, switching SIMs. services is slightly further down the line, with
deployments happen any faster necessarily,
Comms units will also need to be replaced Telenet signaling its intention to switch off
but it will be economic for operators to
and this could be an expensive pain. So, the its 3G networks by 2024, as part of a phased
deploy.”
pace at which 2G actually gets retired will shutdown, while Orange Belgium plans to
He adds that with the shutting down of be a function of the political weight of the switch off 3G a year later, in 2025, with 2G
these legacy systems, operators will have a electricity companies in their respective networks to be turned off by 2030 by the
greater focus on delivering 5G, ultimately countries.” latest.
making the most out of this next-generation
Either way, it remains to be seen what full
technology.
benefits 5G will have once operators are able
Impact on IoT to redistribute the 2G and 3G spectrum.

With IoT products such as smart meters, As Bullock says, it won’t be an instant
3G going first
water meters, and tracking meters still using process and won’t necessarily happen
With many operators preparing to phase out 2G networks, Bullock noted it will have a big everywhere at once.
the legacy networks, it’s actually 3G, which impact on this segment of the industry.
But it’s an exciting time for the industry
came later, that is being phased out earlier
“A lot of businesses still depend on 2G, and could be vital in unleashing the true
than 2G.
with a lot of devices out there needing these potential of 5G. Users just need to make sure
Surprisingly, 3G is less in use right now services, so there could be a serious business they are prepared for when their networks
than 2G - although there are still those impact once 2G is switched off.” may shut down. 

Issue 46 • November 2022 | 47

DCD Magazine #46 >>CONTENTS

Taking down the

infrastructure of
cybercrime
Data centers raids are rare, but cybercriminals
love hijacking legit cloud instances Dan Swinhoe
News Editor

W
hile data centers will “If we're doing a search warrant at a threat RAM, 94 CPU cores, and 620GB of storage.
often shout about the actor residence then it's a little different, A similar example was Kim Dotcom
strict security around but that is very rare these days. It's not very of Megaupload. The New Zealand
their perimeters, and Hollywood at all.” Police arrested Dotcom and three other
in some cases even However, while most raids are done Megaupload executives at a mansion outside
point to the presence quietly, there have been examples of major Auckland in 2012. Reports suggest dozens
of armed guards, it's rare there’s anything law enforcement activity at data centers of armed police swooped on the estate in
close to conflict occurring. And in the few through the years, as well as quieter searches helicopters around 7am on the morning
times there has been any sort of action that made the press. of Dotcom’s birthday party, including
happening on-site, it’s usually being led by several members of New Zealand’s elite
Most notably, the 'CyberBunker' facility
law enforcement and without resistance. counter-terrorist force. Dotcom remains in
in Traben-Trarbach, western Germany,
But while data centers are rarely the stuff was raided by more than 600 police officers New Zealand and continues to operate the
of action films, they are regularly the source in September 2019. Eight people were successor site Mega. Mathias Ortmann and
of illegal and nefarious activity. And the convicted in 2021. Bram van der Kolk, who were both arrested
move to cloud is making it much harder for during the 2012 raid, recently reached a deal
Built by the West German military in the
law enforcement to track down and take out that will see them avoid being extradited to
1970s, the site was used by the Bundeswehr’s
the infrastructure of cybercrime. the States in exchange for facing charges in
meteorological division until 2012. A year
New Zealand.
later, it was sold to Herman-Johan Xennt,
who told locals he would build a web- In 2014, the US Drug Enforcement
Data center raids: Rarely Administration (DEA) and Internal Revenue
hosting business there. Illegal services
Hollywood fodder allegedly hosted at the German data center Service (IRS) agents raided an Albuquerque,
While data center raids are fairly common, were Cannabis Road, Fraudsters, Flugsvamp, New Mexico data center run by a local
they are usually quiet affairs with little fuss. Flight Vamp 2.0, orangechemicals, and provider called Big Byte. The DEA also
A couple of agents or officers with a warrant what was then the world's second-largest searched the Pagosa Springs resort in
are more likely than a SWAT team breaking narcotics marketplace, Wall Street Market. Albuquerque also owned by the same
down the door. family. No arrests were made at the facility,
While less malicious than drugs, Swedish
which is still in operation today. No charges
“Search warrants, or raids at hosting Police raided the Pirate Bay more than once
were brought against the owners, though
providers, are really not all that glamorous, in an effort to take the site down, including
a relative of the owners pleaded guilty to
to be honest,” says Matt Swenson, Division once in 2006 when some 65 Swedish police
submitting a false federal income tax.
Chief of the Cyber Division at the Homeland officers entered a data center in Stockholm,
Security Investigations Cyber Crime Center. and again in 2014. During the 2006 raid, In 2011, the FBI raided a colocation site
“You usually just go into a data center with servers belonging to a number of other in Virginia – reported at the time as possibly
a search warrant that says, you're legally companies, including a Russian opposition CoreSite’s facility in Reston – in search of
authorized to search XYZ server, and the news agency and GameSwitch, a British servers being used to hack into CIA and
provider will find where that's being that's game server host, were seized. The site is still other major institutions and corporations.
being hosted. We’ll then make a copy of the in operation today. Apparently at the time The agency seized servers of Switzerland-
data and take it with us. And we do that fairly of the 2014 raid the Pirate Bay required just based hosting firm DigitalOne.
regularly. 21 virtual machines (VMs) to run; 182GB of The same year, Dallas-based Tailor Made

48 | DCD Magazine • datacenterdynamics.com

>>CONTENTS Raiding data centers 

Servers were raided in hopes of finding two scanners that should detect this type Difficult, but takedowns do
initiators of that month's cyber attacks on of malware, but did cooperate in the
occur
PayPal. As part of the same investigation, investigation and gave Argos access to the
German police executed a warrant for a information on the server. They have since While difficult and time-consuming, major
search of a German hosting company's stopped working with the tenant previously takedowns of illegal infrastructure does
utilizing that machine. happen.
offices.
“We see stuff hosted at gig providers Last year four men pleaded guilty
Most recently in October 2021, police in
like AWS and DigitalOcean. We see a lot of in the US to conspiring to engage in a
South Korea raided an SK Corp data center
infrastructure hosted at big Internet service Racketeer Influenced Corrupt Organization
that had recently suffered a major fire. Local
providers like OVHcloud in the UK and (RICO) and face 20 years in prison for
police confiscated documents relating to
France and throughout Europe, and a lot providing bulletproof hosting services
the fire, which was caused by a battery and
smaller providers that are being utilized,” to cybercriminals. According to the DOJ,
brought down the KakaoTalk messaging
explains Swenson. “You name it, these guys between 2008 and 2015 the group rented
service and disrupted much of the country.
will utilize it. ” Internet Protocol (IP) addresses, servers,
and domains from which cybercriminals
He says that ‘most of the large companies’
Working with law enforcement are very cooperative and respond to the vast
conducted attacks, including malware
distribution, botnets, and banking trojans.
to bring cybercrime majority of legal processes. However, the Malware hosted by the organization
infrastructure down international nature of cybercrime means included Zeus, SpyEye, Citadel, and the
Most cloud and colocation providers take US law enforcement often has to deal with Blackhole Exploit Kit.
little interest in what their customers actually actors and infrastructure based abroad,
Operation Onymous was a concerted
do with the hardware or instances in a which can complicate issues.
effort by agencies including the FBI,
provider’s facility, and even providers in “When we're working a case within the Homeland Security, and Europol to take on
major data center markets can be used to United States, and infrastructures being darkweb markets. Through the cooperation
with police forces of 17 countries, notorious
markets including Silk Road 2.0, Cloud 9,
and Hydra were taken down.
Artem Vaulin, founder of KickAss Torrents,
was arrested after investigators cross-
referenced an IP address he used for an
iTunes transaction with an IP address used
to log into KAT's Facebook page. The FBI also
posed as an advertiser and obtained details of
a bank account associated with the site.
While most the of world is looking at
ways to lengthen the lifecycle of hardware
and reuse the likes of servers, in the wake of
raids and seizures little hardware survives
once the investigation is over.
“If it's used in the commission of a crime
it'll be wiped and destroyed,” says Swenson.
“Way back when I first started and kind of
the early 2000s, we used to wipe a lot of
computers and then repurpose them. But
host cyber criminal infrastructure. hosted abroad, we rely on the cooperation we moved away from that. So almost all of it
Last year an Iranian malware campaign of foreign governments to respond to legal now gets wiped and destroyed.”
attacking targets across the world was found process,” says Swenson. “But the process
He says one of the reasons for this is
to be being hosted out Dutch colocation data is not fast, particularly abroad, and a lot of
precautionary security reasons, in case
centers. Cyber firm BitDefender found the times we don't have months to wait.
hardware has particularly resilient malware
command and control (C2) infrastructure of “If that country doesn't respond or isn't present that may be able to survive a
two strains of malware linked to Iranian- responsive to the US legal process, there's hardware wipe.
attributed Advanced Persistent Threat nothing we can really do in order to to get
(APT) actors were being hosted within the a copy of that server. A lot of infrastructure
Netherlands. The server was being hosted is being hosted in Russia and Belarus, and Crypto: dangerous but useful
by American hosting company Monstermeg, we just can't get a lot of cooperation. A Crytomining can be profitable but dangerous
which provides services out of Evoswitch’s lot of cyber criminals know that, so they for criminals. A shootout at a cryptomining
AMS1 Amsterdam data center in Haarlem, specifically stand up infrastructure in data center in Abkhazia, a separatist state
and the malware had been present there countries that are untouchable by US law recognized as part of Georgia, led to one
since April 2020. enforcement.” man being killed during an attempted
Monstermeg owner Kevin Kopp told Swenson does note, however, that the robbery by armed gunmen.
Argos the company was not aware that FSB will cooperate with the US if it's a child In February 2021, Spain's national police
this malware was on the server, despite exploitation investigation online. raided a building that they thought was

Issue 46 • November 2022 | 49

DCD Magazine #46 >>CONTENTS

being used to grow marijuana, with the Russian government, used Google “Cyber criminals needs need a place to
that it was an illegal cryptocurrency Drive cloud storage services as well as store their software and a safe environment
mining operation. Dropbox, a company that transitioned off to distribute them,” says Fulton. “And those
“We have seized a lot of equipment the cloud back to its own data centers. In could be cloud, EC2 instances, S3 buckets,
being utilized to mine for crypto,” explains 2019 and 2020, RiskIQ (since acquired by for example, that are never well monitored;
Swenson. “A lot of times dark web criminals Microsoft) reported that Magecart credit they'll find universities and non-profits and
card-skimming attacks were repeatedly large enterprises that don't control their
will have a side business where they are
being launched from poorly-configured sprawl, and they'll squat there in order to
cryptomining. But it's not as common as it
Amazon Web Services Simple Storage assemble the kits, practice their exploits,
was a few years back, I think the hobbyists
Service (AWS S3) buckets. According to execute them on unmonitored systems and
have kind of been pushed out.”
Malwarebytes, malware delivered over the refine the tool.”
More common, however, is the use
cloud increased by 68 percent in 2021. He says criminals are also increasingly
of cryptocurrency to pay for hosting
Lumen’s research arm Black Lotus Labs using short-lived cloud instances from
and obfuscate their identity if there is an
recently published research that points to hijacked legitimate accounts to probe and
investigation by law enforcement.
more than 12,000 servers that are running scan network perimeters and defenses, and
“We see a lot of movement to then launch attacks.
Microsoft domain controllers hosting
the payment of infrastructure via
the company’s Active Directory services “The attackers who make use of the cloud,
cryptocurrency,” says Swenson.
and regularly used to magnify the size of do so because it makes them a continuously
“A lot of the hosting providers are now distributed-denial-of-service (DDoS) attacks. moving target,” says Fulton. “With auto-
accepting various forms of cryptocurrency scaling groups, elastic responsiveness can
Such attacks are called ‘living off the
and that can add a layer of anonymity be 20,000 or more computers, spins them
land’ attacks and can be harder to spot and
because they no longer have to provide a up in seconds and sometimes they last just
stop as companies often whitelist legitimate
credit card or a bank account; they can just minutes. And enterprises don't have the
companies such as Google, Amazon, and
move it from a wallet that's been completely ability to know that all 20,000 are theirs or
Microsoft. Access to cloud accounts with
stood up without any sort of information what is on them.
credits already in hand to procure more
that can be used for threat actor attribution. “If, for instance, a ‘legitimate’ server that
compute resources can be sold for a high
“The hosting providers, they're in it to price, reports IBM. only exists for three minutes probes you for
make money. And I don't necessarily think vulnerabilities, it's fast, nobody can notice
Coalfire’s Barratt says it's not uncommon
their number one concern is who's paying it. I would use one of those short-lived
to see cloud accounts hijacked and used
the bill. I don't know that they really care all instances to collect my tools and pre-
to mine cryptocurrency. A 2021 report
that much because they're usually going to position them. ”
from Google said: “86 percent of the
do the bare minimum that they have to do That move to the cloud has changed how
compromised Google Cloud instances were
in order to be compliant.” law enforcement approach and deal with
used to perform cryptocurrency mining, a
investigations, and seen a massive shift in
cloud resource-intensive for-profit activity.”
the types of devices seized during raids and
Cybercrime moves to the cloud The NSO Group, which is less a investigations.
While malware and cybercrime cybercrime group and more a state-
“I started as a digital forensic analyst in
infrastructure continues to live in physical sponsored hacking company for hire, was
the mid-2000s and there was no cloud back
data centers, much of it has been abstracted previously hosted out of AWS infrastructure
then. Everything was stored locally, and we
and virtualized to the cloud. And in the same until it was kicked off the platform in
would see a lot of external drives and stuff
way legitimate enterprises are looking to the the wake of an Amnesty report into its
like that,” says Swenson. Nowadays that it's
cloud to reduce the amount of on-premise operations. NSO’s Pegasus spyware is used
almost the opposite, and a very minimal
hardware they need to manage, criminals by numerous governments around the world
amount of data being stored locally.
are copying that trend. to spy on media, opposition political figures,
activists and NGO workers, diplomats, and “Where we used to go in and seize a
“[The German facility] is the only ‘illegal bunch of computer towers, now it's a lot of
others. NSO is also known to use Digital
data center’ I've personally seen and heard iPads, Chromebooks, and phones that are
of in the physical sense,” says Andrew Ocean, Linode, OVHCloud, UpCloud,
Neterra, Aruba, Choopa, CloudSigma. then connecting to the cloud, and they're
Barratt, Principal Consultant of Adversary not storing anything locally.”
Ops at penetration testing firm Coalfire. The difficulty enterprises face managing
That change has made investigations far
“And I suspect because it's just really hard virtualized, multi-cloud, and increasingly
more difficult for law enforcement from a
to do and go unnoticed; there's loads of just serverless infrastructure is also creating
legal perspective, as cloud-hosted data can
really dull logistical stuff that make it hard huge opportunities for cybercriminals.
often escape warrants.
to run physically dark operations without “Most enterprises lack awareness of
making yourself a huge red flag to lots of “[The cloud] has made things more
what's in their environment, or even where
people very quickly. problematic for us from a legal perspective:
their crown jewels sprawl to,” says Joel
If I have a search warrant for a house and
“But we've seen that the more Fulton, CEO of security startup Lucidum
computers in a residence, I don't necessarily
sophisticated intruders are heavily and previously CISO of Splunk. “And the bad
have the authority to grab the data from
leveraging compromised cloud guys are building infrastructure now so that
a cloud provider because it doesn't exist
environments where their approach is more it can be transitory.”
at the actual physical residence. Either
about building up virtual data centers that This combination of cloud-enabled write a separate warrant for cloud storage
can leverage infrastructure that they don't sprawl and increasingly ephemeral or add it to a warrant if we're going into a
have to pay for.” infrastructure is providing a safe haven residence. It's just a matter of figuring out
Threat reports from Unit42 suggest from which attackers can develop, store, and where that's being hosted and then adding
Cloaked Ursa, a threat actor group affiliated launch attacks. additional legal process into what we do.” 

50 | DCD Magazine • datacenterdynamics.com

>>CONTENTS Ampere's core challenge 

Ampere's core
challenge Sebastian Moss
Sebastian
Editor-in-Chief
Editor-in-Chief

Building an Arm chip

for the cloud

T
"
he story of Arm in the data
center isn’t new, that’s not
an important detail. The real
story is that we've hit a tipping
point where the cloud requires
something new because of
the way it runs, and it needs to be efficient.
Nobody who’s doing x86 is building that. It
just so happens that we’re Arm-based, but
we are building it.”

Over a wide-ranging discussion in

London, Ampere Computing's chief product
officer Jeff Wittich gave DCD a detailed
run-through of the company’s technology
roadmap, and why he thinks the five-year-
old business can dominate the cloud CPU
market.

Arm’s multiple attempts to break into

the data center may not be new, but quickly
revisiting the tale provides valuable context.

You had the early failures that just weren't

ready," Wittich recalls. "The Calxeda chips,
the Applied Micro stuff, but the model was
wrong, people thought that you could take
small cell phone chips and if you scaled
them enough out, you would be you would
have enough performance. The problem
is, is that there is a minimum bar for
performance per core, it's not zero."

But they laid the groundwork, and got the

ball rolling on a software ecosystem.

Next came the middle phase, where

"a bunch of people that had the wrong
approach and or got out too early, when it
got hard, because it was a side bet as part of a
differentiated business."

One such example is Qualcomm's Arm

server processor line. Its Centriq 2400 chip
was well received, and may have proved

Issue 46 • November 2022 | 51

DCD Magazine #46 >>CONTENTS

features that make complete sense for a

client processor but make no sense in the
cloud."

Its cores, it argues, are targeted directly

at the cloud. Even traditional high-
performance computing is too tangential a
market. "I'm not focused on that, you would
make a different type of core."

There is one other market that Ampere is

targeting beyond cloud and on-prem cloud,
though: The high end Edge.

"The self-driving car company Cruise

uses us in their vehicles,” Wittich reveals.
“This isn't us getting into the automotive
space, more that they needed a really high
performance Edge server that would sit
in the car. They couldn't actually find any
successful in the long run. But then the would not dissuade users. For Ampere to other CPUs that within 100 watts that gives
company got into a legal fight with its succeed, it needs to show a clear advantage, a reasonable performance, and our 64 core
largest customer, Apple, and spent most of Wittich admits. chip consumes 70 watts."
2018 fighting off a hostile takeover attempt But the company hopes that cracking
That's where its big bet on cores comes in.
by Broadcom. the automotive space will tie back to the
UK-based Arm licenses out its cloud. "There's a bunch of other smaller Arm
Bruised and in need of reassuring
eponymous instruction set architecture to devices sitting everywhere in the vehicle,
investors with cost cutting, it laid off
chip designers. It also licenses out processor but a lot of the developers are just doing that
hundreds from its data center division and
core technologies, with the latest being stuff on x86 machines in the cloud, and then
killed the project. Ampere and Microsoft
Neoverse. moving it over to the car. You're constantly
hired many of those let go.
porting back and forth. And it's a waste.”
That's what AWS relies on for Graviton.
But now, after years of failures, the stars
And it's what Ampere used to rely on. Building its own cores also reduces its
finally seem aligned. Amazon Web Services
licensing fees to Arm, and insulates it from
released its Graviton series of chips, while "On Altra & Altra Max, we decided that we
the chip designer's ups and downs. In late
the fastest supercomputer of 2021 uses couldn’t sit around for five years building a
2020, Nvidia announced it would acquire
Fujitsu's Arm A64FX processor. core,” he says. “We said ‘let's get something Arm for $40 billion but, after two years of
out because otherwise we won't have distracting regulatory investigations, the
"The software ecosystem is there, so stuff
is either at parity or in some cases even a customers, we won’t have feedback, and we deal collapsed.
little bit ahead of where the x86 stuff is," says won’t have an ecosystem.
Arm soon appointed a new CEO, who
Wittich. "The time is now." “Going forward, with what we're doing announced layoffs and plans for an IPO.
Of course, once you have come to the with our cores, it looks a lot different. And "I didn't have to worry that much during
conclusion that Arm is inevitable due to its it starts to really deliver huge density rate all this stuff over the last two years," says
energy efficiency and flexibility over x86, power efficiency, while delivering the type Wittich. "We have an architectural license,
the next question has to be which form of of performance that the cloud wants. and we can build what we want to build."
Arm. "I just don't know that we'll see that from But Ampere is not alone in that, with
With AWS several generations into its Arm [cores].” others developing their own cores.
own processor family, Microsoft known Ampere's view is that Arm's own cores On the consumer side, Apple took the
to be working on its own Arm chips, and will never put the data center first. "Arm approach for its M1 Arm chips, first launched
Google likely cooking up something, what develops the cores for the client product in 2020. On the data center side, things get a
space is there for Ampere? first, and then they adapt them to little more complex.
"They're big companies,” Wittich admits. infrastructure cores a few months later. But
at their heart, they were still developed for a In 2018, three Apple veterans launched
“But our customers are a blend of all those Nuvia to build their own Arm server CPU
companies, so we can get to massive different market."
with their own cores. Just a year later, Apple
scale. It’s also useful from a network effect That means a lot of other approaches to sued one of the founders claiming that he
perspective. As an end user, is it possible Arm server CPUs are flawed, Wittich argues. solely worked at Nuvia while still employed
to be running on Graviton and the Ampere "It's one of the fundamental problems with a by Apple.
Altra processor across two clouds? Sure. bunch of the CPU models today, they have
"But it's a level of complexity that doesn't
need to exist when we can come in and run
the same processor at Microsoft, at Google, “This isn't the old days of the Arm chips that
or on an HPE box.” come in and just undercut everybody on price,
Still, if a hyperscaler-made processor was
significantly better the lack of network effort
we're not the lowest price"

52 | DCD Magazine • datacenterdynamics.com

>>CONTENTS All about the core 

In 2021, it seemed like that Nuvia had

put the controversy behind it, selling to
Qualcomm for a respectable $1.4 billion. "A bunch of the [Arm] CPU models today have
Curiously, it seemed like it also put the server
chip behind it, with Qualcomm announcing
features that make complete sense for a client
that it would use the tech in mobile, IoT, processor but make no sense in the cloud"
and networking products, integrating it into
Snapdragon.

A year later, it pivoted back, shopping

around a server product. Over the summer,

Bloomberg reported that Amazon agreed to really, really consistent,” he said. “We don't With x86, “the problem with it is that
look at the chip. want any variability across the chip, where you're getting into a space where that
if you got placed in one core versus another additional performance gets really power
Yet more confusion soon followed. In inefficient,” Wittich says. “So you're adding
core, the performance looks a lot different.
September, Arm sued Qualcomm - one of 20 percent more power to get 10 percent
its largest partners - claiming that it did not "We wanted to avoid any bottlenecks. A more performance.”
agree to Qualcomm’s use of Nuvia’s licenses, lot of the chiplet approaches to date have
and terminated the licenses in February. big bottlenecks, because there are too many At the rack and data center level, that
doesn’t make sense, he says. “Each chip
hops and the latency is still too large from
Should it win the case, it could unwind a looks like it's delivering more performance,
chip to chip.
major acquisition for Qualcomm, and wreck but overall you've just reduced your overall
its desktop and server chip plans. Even if the “Our chiplets interconnect is done in capacity. For no reason.”
case is ultimately settled, it will delay and such a way that we remove a lot of the
Still, while he eyes those x86 workloads
distract Nuvia - and it's hard to have faith common bottlenecks that occur in a chiplet-
as land to conquer, Wittich notes that he’s
in Qualcomm's management to maintain based approach,” he claims. aware of where the Ampere chips’ limits are.
focus.
The company was able to get 128 cores “Trying to make everything one size
Wittich is diplomatic in his views on into a single die; it plans several hundred as fits all is a disaster. We do awesome at
Nuvia. "We have our own cores, and that it goes chiplet. “We want to make sure that as inferencing on a CPU, but if you’ve got
gives us a five-year lead over anyone you bring more and more cores online, that batch inference jobs that you're gonna
who decides it might be time to start the performance per core doesn't go down. plow through over the next 12 hours,
designing their own cores. Now, that's a big maybe move that stuff off to an inference
And that's not really the case with a lot of
differentiator." accelerator.”
legacy x86 CPUs.”
Another factor it hopes will give it a lead “I don't think we're going back to a
That’s why this is not a story about Arm,
over Arm and non-Arm processors is its own market where you've got one CPU that's
he argues. “This isn't the old days of the
chip-to-chip interconnect, which will let it deployed at 99 percent of the servers out
Arm chips that come in and just undercut
go to a chiplet approach, where tiny dies are there, we're not going back,” says Wittich,
everybody on price, we're not the lowest
used instead of one monolithic die. who was lured to the company after 15 years
price. But what we are is for the highest
at Intel watching that market share fall. “The
“Our first two products went monolithic, performance processor, and we're the most
world's changed now.” 
because it’s critical that our performance is power efficient processor.”

Issue 46 • November 2022 | 53

DCD Magazine #46 >>CONTENTS

The UK’s next

Paul Lipscombe
Telco Editor
big telco merger

Will four become three in the UK?

Advanced talks between Vodafone
and Three have got the industry
talking about a potential merger
and its chances of success

T
he telecoms market loves to consolidate.

We can see it in Europe, where Orange and

MásMóvil are set to combine in Spain, and in
South Africa where MTN and Rain have been
tussling over a merger deal with Telkom, with the
former recently pulling out of talks.

And we saw T-Mobile merge with Sprint in 2020, in a bid

to compete with AT&T and Verizon in the US.

Nick Read The next big potential merger on the line could be
Vodafone CEO between Vodafone and Three, in the UK.

There have been rumblings of this potential merger for

a while, but in late September, Vodafone confirmed both
operators are in advanced talks over a deal, which could
potentially value a merger deal between £12 billion to £15bn
($13.4bn to $16.8bn).

In a statement, Vodafone said the envisaged transaction

would see Vodafone owning 51 percent of the business,
with conglomerate CK Hutchison owning the remaining 49
percent.

"By combining our businesses, Vodafone UK and Three

UK will gain the necessary scale to be able to accelerate
the rollout of full 5G in the UK and expand broadband

54 | DCD Magazine • datacenterdynamics.com

>>CONTENTS There can only be one 

connectivity to rural communities and small

“They do make good complementary Sheppard points to a merger between
businesses," Vodafone said in a statement at
partners. Vodafone is more premium than Vodafone and Three in Australia, which
the time.
Three and has a strong enterprise base provides evidence that a merger between the
The operator also added that the “merged compared to Three’s limited enterprise base. two groups can be successful in the UK.
business would challenge the two already
“Three has a more youthful brand than “From my viewing of what I’ve seen it
consolidated players for all UK customers.”
Vodafone and it appeals to younger data- was a tricky project to integrate the networks
Vodafone is referring to EE, which hungry consumers who are less likely to be at the time, and had some challenges
merged with BT in 2016, while EE itself was found on the Vodafone base.” combining the two networks,” explains
created as Everything Everywhere back in Sheppard.
2012, when Orange and T-Mobile merged. “But once these challenges were
The other big operator in the UK is
Why now? overcome, it’s become a scalable enough
of course O2, which merged with Virgin network for the company. It’s successfully
Understanding how this deal came to be first
Media in a £31.4bn ($35.3bn) deal last year, managed to keep trading in the country and
requires understanding the UK telecoms
in another fusion of mobile and broadband has a long-term vision.”
market. The mergers between EE and BT,
assets where the merger was split 50:50. and O2 and Virgin Media, have seen these
So, unlike the other two previously companies increase their connection
numbers drastically. 5G is a big asset
mentioned mergers, this would be different
in the sense that it would connect two After combining forces last year, Virgin It’s worth noting that Three UK has a
mobile-specific operators, although Media O2 now has more than 47 million solid asset at its disposal, and that’s the 5G
Vodafone does boast a sizeable broadband customers across broadband, mobile, phone, spectrum the operator has picked up in
business. and home subscribers. recent years.

Meanwhile, BT and EE, along with The company has acquired 160MHz of 5G
broadband provider Plusnet, which was spectrum in total, with 100MHz of this in a
An obvious move acquired by BT Group in 2007, boast 35 contiguous block.
"Confirmation of a potential tie-up between million subscribers, across mobile and As a result, Three claims to cover 56
Vodafone and Three comes as no surprise broadband services. percent of the UK with its 5G coverage, as
- the two companies have made no secret of July 2022. This spans over 400 locations
In comparison, Vodafone has over 18
of their interest to consolidate,” says across 3,200 locations.
million subscribers, although most of which
CCS Insight’s director for consumer and
are mobile, bar about 600,000 broadband Sheppard acknowledges Three’s 5G
connectivity Kester Mann.
customers. Three UK has about half this spectrum gains, and notes that Vodafone
"The leading motivation to join forces amount, with around 9.3 million customers. has a good stock of spectrum in the lower-
is scale. In telecommunications, the most frequency bands as well.
Although the number of connections
successful companies tend to be the largest;
would still be less than the other big two “I’d say that all the operators have a
bulking up would offer many synergies and
operators, the number of mobile subscribers decent amount of spectrum, so I don’t think
cost-saving opportunities. Under the status
will make up the bulk of the combined 27 the merger is being done purely for this
quo, it’s hard to see either operator growing
million plus customers. basis,” he said.
enough organically to challenge BT and
Virgin Media O2 for size in the UK." Phil Sheppard, formerly director of “Three’s 5G spectrum holding is very
network strategy and architecture of Three good, slightly better than the others, but then
Meanwhile, James Gray, managing
UK and now a self-employed telecoms again Vodafone’s low-frequency spectrum
director of Graystone Strategy, says the
consultant, says it’s in the interest of both holding, which provides the wide area
ramped-up discussions and talk of the
operators to merge, noting that Three UK network (WAN) coverage, is better, so I think
merger have been the worst-kept secret in
especially could benefit from the scale that a the combination is good.”
the industry.
merger would bring.
“With the potential merger of Three
Gray is uniquely positioned to comment
“Both parties need this,” he says. “The and Vodafone, the joint company will
on these talks as he’s worked for both
public analysis of Vodafone is that its UK have access to a large spectrum asset,” said
companies. At Vodafone, he worked across
business is an underperforming market NTT Data UK & Ireland president head of
a range of marketing roles for a decade
compared to some of its other markets networks Sharad Sharma.
between 2002 and 2012. More recently he
globally and so Vodafone needs to do
was a marketing strategy consultant at Three “This will enable a joint venture to take
something about this. As for Three, it’s the
for close to three years until 2020. some technologically advanced initiatives
smallest operator so it desperately needs
around 5G and later 6G, putting the UK at the
He agrees with Mann that a merger would some scale in order to perform better. I do
forefront of tech innovation.
make sense for both operators. think it works for both.”
“Moreover, with the joined-up might of
two large organizations, we expect to see
the expedited rollout of 5G networks and
“Three’s 5G spectrum holding is very good, but broadband to rural areas. Vital services will
then again Vodafone’s low-frequency spectrum benefit from faster connectivity – enabling
faster response times, better patient care, and
holding is better" more reliable service all across the public
sector in underserved areas.”

Issue 46 • November 2022 | 55

DCD Magazine #46 >>CONTENTS

to be worth $23bn once it was finalized in

2020.

This merger has been somewhat of a

success, if you note that T-Mobile, which the
company is still called, has been driving its
5G deployment, with the operator claiming
to cover more than 5,000 cities nationwide
with 5G coverage.

The company is also clearly number one

for 5G coverage in the US against its rival
operators for area coverage, with a report
by WhistleOut revealing that its 5G footprint
extends to 53 percent of the country. In
comparison rival networks AT&T and Verizon
are reportedly lagging behind with 29
percent and 12 percent coverage respectively.

Mann believes that the merger of

T-Mobile and Sprint could be a useful
blueprint for Vodafone and Three to follow,
as the US operators shrunk down from four
MNOs to three.

“T-Mobile is on a roll at the moment. It

has all that great mid-band spectrum from
Sprint, and has been able to roll it out fast.
It’s definitely putting pressure on Verizon
and AT&T and that’s good for the industry
“The complex network sharing deals could be an because it puts pressure on the incumbents. I
think so far, this is a great example of what a
issue, notably as Vodafone shares with O2, and successful merger can be.”
Three shares with EE" It should also be noted, however, that
T-Mobile has laid off a significant number of
employees in recent times, something that
then-CEO Legere said the company wouldn’t
do.
Vodafone under pressure to Vodafone Group is also looking to sell a
stake in its Vantage Towers business unit, Little is known about what Vodafone
perform and Three’s plans would be, but any merger
with several potential buyers reportedly
A merger has been on the horizon for some lining up bids, including American Towers, would see a lot of empty retail spaces on
time for Vodafone, with its chief executive Cellnex, Brookfield Asset Management, and the high street, as four operators dilute
Nick Read coming under considerable Digital Bridge Group, while a sale of a stake into three. Job losses are likely, which the
pressure from shareholders in the past could fetch the company £12bn ($12.83bn). company will say in the name of improving
year following disappointing financial efficiency - but if too aggressive could cause
performances. Meanwhile, Three’s CEO Robert Finnegan problems.
has made no secret of his views on a merger
It’s no secret that Vodafone is keen Sheppard is keen on the idea of
deal, and consolidation for the UK market.
to streamline its business, with a recent consolidation, but only if the impact on
He’s previously said that the UK market
consumers is positive.
investment from French tycoon Xavier Niel, has too many players, and has repeatedly
who purchased a 2.5 percent stake in the referred to the market as “dysfunctional.”
company.
Approval will be needed
Niel, who is a founder and major
shareholder in French telco Iliad, is keen T-Mobile US blueprint? Before any merger deal is completed,
to pursue consolidation opportunities, there’s a fair bit of scrutiny that needs to
But can consolidation be successful for both be done, especially around the fairness of
further driving speculation that a merger
operators? Even if this does go through, both competition, and all three people DCD spoke
between Vodafone and Three is likely to
operators will continue to lag behind the to agreed that the Competitions and Market’s
happen.
other two for customer numbers. Authority (CMA) and UK regulator Ofcom will
The operator recently acquired have a lot to look at.
Still, a similar situation occurred in the
Portuguese operator Nowo from MasMovil,
US, where the third and fourth biggest “Should any deal materialize, regulation
but has sold off large chunks of business
operators at the time, T-Mobile and Sprint, would be a major hurdle,” says Mann.
in other markets, notably its Hungarian
merged.
business unit for $1.8bn in August, its “It would be up to the competition
Egyptian unit to Vodacom (in which it owns The merger was first proposed in 2018, authorities to decide whether reducing
a large stake), and is reportedly in talks to sell by the then T-Mobile CEO John Legere and the number of players is for the overall
its Ghana business to Telecel Group. Sprint CEO Marcelo Claure. It was estimated good of the market. Advocates will argue it

56 | DCD Magazine • datacenterdynamics.com

 There can only be one 

encourages investment; dissenters will claim interests and more sustainable for the market for the other operators to step in and get
it’s a reason to push up prices.” if the two operators merge. some customers during this period if the
customer experience drops during this
Just a few years ago in 2016, a proposed Sheppard points out another potential
merging process,” adds Gray.
£10.45bn ($11.7bn) merger between O2 and barrier to a deal, the shared network deals
Three was knocked back by the European that the operators have in place. Mann expects that the deal will likely go
Commission, as it would have reduced ahead, as does Sheppard.
“The complex network sharing deals
competition in the UK mobile network
could be an issue, notably as Vodafone “If I was to guess on this, I’d say yes, (on
operators market.
shares with O2, and Three shares with EE, whether a deal will happen),” said Sheppard.
The decision was overturned in 2020, and this involves sharing different pieces of
“However I’d expect it to be heavily
when CK Hutchison, the parent company of networks, sites, sharing arrangements, and
scrutinized and there will be some
Three appealed that there was no evidence suppliers. They’ll need to work out what to do
mitigations put in there that might be
that a merger would hurt competition. with that in order for a deal to be approved,”
complex and difficult. By mitigation, I mean
says Sheppard.
At the time the merger was blocked, and obligations to the other telcos as there are
O2 has since merged with another company. The market has consolidated a lot in network-sharing deals and potentially some
Gray, though, believes this time it will be recent years, even if there has been some concessions. It should get through this time.”
different, noting that the UK is no longer in resistance to the idea of slimming down
The Virgin Media and O2 merger was
the EU, while the appetite for consolidation choice.
provisionally cleared by the CMA within 12
is more appetizing in general.
However, there’s been successful months of the deal first being announced. It
“The CMA and Ofcom will want to put a showings of consolidation, notably the remains to be seen if the same will happen
lot of scrutiny on this, as the companies are previously mentioned T-Mobile and Sprint for Vodafone and Three.
very similar mobile offering businesses. It’s merger.
Both parties can take encouragement,
different from the Virgin and O2 merger,
Gray believes that, although the other with Thailand’s regulators approving a recent
with one business being more broadband
UK operators, EE and VirginMedia O2 won’t merger between DTAC and True, and this is
and the other mobile-focused.
necessarily welcome a merger, it wouldn’t one that effectively leaves just two operators
“However, the UK is no longer part of have come as a surprise to them either. jostling for position.
the EU and that deal was referred to the EU
“They will have watched the speculation, However, this October Virgin Media O2
for review. This would be a decision made
much like we have, and they’d have planned pulled out of plans to acquire TalkTalk due to
largely in the UK, while this EU decision
how they will react to this when it happens. market and regulatory uncertainties, with the
actually was overturned.”
deal thought to be worth £3 billion ($3.44bn).
“There may even be a short-term
Gray expects there to be a less aggressive
benefit for the other operators as Three It remains to be seen what the outcome
approach from regulators over a potential
and Vodafone will be somewhat distracted will be, but for Vodafone and Three, it might
merger, adding the amount of investment
initially as they attempt to merge, it will take just be a necessary fit to challenge the other
needed for 5G for all the operators is
some time and this could be an opportunity two players in the UK. 
significant and that maybe it’s in everyone’s

Issue 46 • November 2022 | 57

DCD Magazine #46 >>CONTENTS

Making the most of

your waste heat Sebastian Moss
Editor-in-Chief

After years of hot air, the digital boiler

market is heating up

A
Images by: Qarnot
s data center operators strive
to make their facilities more
efficient, some are embracing the
concept of reusing waste heat.

Greenhouses, fish farms, and apartments

have benefited from the idea - with servers
using the heat that they don't want to warm
those that need it.

But it requires data centers to be near

district heating systems or farms, and is
viewed as a secondary priority after the main
job of being a data center. What if we flip the
idea on its head? What if we find the places
that are already using electricity to create
heat, and stick compute in the middle?

A few companies have tried to build digital

boilers and heaters over the past decade, but
the idea has struggled to find traction, with
several startups going bust or pivoting out
of the sector. Now, French company Qarnot
thinks it has managed to solve the challenge.

"10 years ago, we were considered to be

crazy," Paul Benoit, the company's CEO, told
DCD.

"And what is very interesting is that for a

year now, what we do is considered the way
to go to deploy more IT infrastructure with
less energy," he proclaimed.

It's a bold statement that may slightly

oversell the company's success - many are
still uncertain about the concept - but there
are signs things are changing. In 2020, the
company raised around $6.5 million, adding
to the $2.5m it had received from Data4
Group a few years earlier.

Qarnot also has clients it can point to,

including Société Générale, KTH Royal

58 | DCD Magazine • datacenterdynamics.com

>>CONTENTS Ampere's
Reusing
corewaste
challenge
heat 

Institute of Technology, and animation house crypto-heater launched in 2018, was mostly a To offset the higher costs, Qarnot partners
Illumination (best known for the Minions marketing stunt, Benoit admits. with a heating company "and then we sell
franchise). the heat we produce at a much lower price to
“It was B2C, which we’re not doing
them than they sell the heat themselves."
Before we unpick how it got to those anymore, and it was quite expensive,” he
deals, first let's understand how we got here. said, but added that those that bought it Traditional air-cooled data centers
"In the beginning, we made a space heater for would have nearly doubled their money on produce waste heat at around 30°C (86°F);
houses," Benoit explained. Ethereum mining at the currency’s peak. Qarnot’s boilers use direct water cooling to
Benoit would not disclose how many units deliver water at at 65°C (149°F) through 2cm
The concept was that, instead of using
were sold, but said it was small. “We’re copper pipes.
your electricity to produce heat through a
focused on B2B now.”
traditional radiator, use the same amount
of electricity on a Qarnot box that looks like There's still a limit to housing blocks,
a radiator. At the same time as producing though: "The scaling is not great, because
heat, a side product would be compute, you cannot put 500kW of hot water 24x7 in
which could be sold to other companies as that housing [complex], even if it's a large
distributed computing. one," he said. But Europe is building more
local district heating, covering multiple such
But the economics were tricky as the
properties from one location - "this is our
equipment cost significantly more than
strategic focus."
a comparable heater. There was also a
fundamental flaw: Data centers require 24x7 The hope is to convince such sites to
compute, while people don't require 24x7 use their energy twice, both for heating
heating. There were also security concerns and computing, while paying only for the
about putting servers in people’s homes, and heating. The digital boilers themselves are
limitations to residential fiber connectivity. more expensive than conventional ones,
because you're replacing simple heating
In the Netherlands, these issues led to
equipment with semiconductors and IT
rival e-radiator business Nerdalize declaring
infrastructure. The IT infrastructure also
bankruptcy in 2019.
becomes obsolete much faster than normal
"So now we go to social housing [and boilers do.
large apartment blocks], where we can
"We provide the chassis, and replace the
run 24x7 for hot showers and stuff," Benoit
servers every five years," Benoit said. The
explained. "We're stopping doing heaters and
company works with circular economy firm
going more for boilers,” he said
IT Renew to use recertified and new Open
One exception, a much-publicized Compute Project servers.

Issue 46 • November 2022 | 59

DCD Magazine #46 >>CONTENTS

expected to grow to] 500kW-1MW sites, the

sweet spot for district heating that is 24x7,"
he said.

As Qarnot deployments grow larger,

they bump into a different challenge:
Connectivity. Nobody wants their residential
block's Internet being sapped by their boiler,
and nobody wants to pay for a data center
that is limited to residential fiber rates.

"Historically when we deployed heaters

in the building, we would deploy 100-300 of
them and use fiber to the building," Benoit
said. "Now we're building larger sites, so we
deploy dark fiber because the investment is
reasonable compared to the infrastructure
and it can be financed by the end user."

The company currently rolls out dark fiber

for 500kW and above projects, but plans to
do it for sites in the lower hundreds moving
forwards.

It may also need to build in more

redundancy as its sites grow bigger. "It's
totally possible if clients want it, and some
do, but most do not ask for it," Benoit said.
"We have to be able to understand the needs
of the client and then discuss whether they
Images by: Qarnot
want to pay for it.”

"Megawatts of heat at 30 degrees? It's crap, computing workloads. The next difficulty is finding those sites,
you cannot do anything with it," Benoit said. with district heating systems only available
“Most of our clients are not very sensitive
in some countries. “In France, district heating
At the same time as it sells the heat, to latency, because they are doing simulations,
penetration is very low,” Benoit admitted. “We
Qarnot then sells its compute to customers, at and workloads like that,” Benoit said. There is work in Finland and want to work there more,
a low price because the electricity costs have still an opportunity for placing the boilers in as well as in all the Nordic countries, where
been offset. the same building as clients, or at least nearby, we have partners. We also have discussions
That compute infrastructure, which is but Benoit does not see it as a priority. “It in Canada, and in Japan - but Japan is special
either paid for by the customer upfront or on would be great,” he said. “But the problem is in terms of infrastructure.”
demand, is accessed through interconnection more on the business side, because if you have
to discuss with the heat guy of the building His hope is that, as countries invest in
points at investor Data4's data centers, and
net zero infrastructure, more district heating
managed through a custom software stack. and the IT guy of the company, it may take 10
systems will begin to be rolled out across the
years to make the deal. We don't do this, but it
While the deployments are close to world - at least in the colder regions.
can be a way to go [in the future].”
potential end users, it’s important to note that
“The opportunity is gigantic,” he said.
it is currently not an ‘Edge’ system focused French bank Société Générale is trialing
on low latency. Instead, Qarnot is targeting using the system for risk computation. "It's This feature is from our Critical Power
batch-processing high-performance a first proof of concept, but the project [is supplement: Read the rest for free here. 

60 | DCD Magazine • datacenterdynamics.com

DCD Magazine #45 >>CONTENTS

 Difficulties Staying Up

Brace for impact

T
processors and technologies.
he last decade has been one of
prosperity and growth for the data At the same time, pressures on energy
center sector. Even the pandemic, markets remain heavy, with the Ukraine war
which destroyed other industries, unlikely to end soon.
meant a fresh wave of investment
and expansion as the world rushed Already, the most fragile in the sector have
online. declared bankruptcy: Sungard, UKCloud,
Datacenter Almere.
But now comes trouble. The global economy
is teetering, and the future is growing ever They had preexisting issues that were
more uncertain. How will the data center sector exposed by the worsening times. Others will
fare in a recession? fare better, but they will not be unscathed. Even
the hyperscalers, often seen as endless pits of
The good news is that it’ll do better than money and power, are feeling the pinch. Their
many - core data center services are a necessity, market caps have fallen dramatically, and they
so enterprises won’t be able to drop them warn of lower capex spending - which will filter
entirely. But it would be naive to think that there down through the whole industry.
will be no impact.
A recession is not an existential threat for the
As cost-cutting becomes a priority for sector, but it is a time to be cautious. The dot-
businesses, expect them to drop non-essential com bubble brought Equinix to its knees, and
workloads and expansions. Startups will slow, led to the end of Exodus Communications.
relying less on unsustainable business models
that leverage the cloud to scale rapidly. Equinix recovered, prioritizing a sustainable
business model over the excess of the early
On the building side, debt markets will not Internet era. Others would be wise to take note.
be as friendly as they once were, investors will 
not be as abundant. Chip fab manufacturing
will be scaled back, slowing innovation in new - Sebastian Moss, Editor-in-Chief

62 | DCD Magazine • datacenterdynamics.com

NEW PILLER UB-V SERIES UPS:
The Power to Protect
piller.com

Nothing protects quite like Piller

Electrically coupled UPS for protection of industrial process through to large scale data centres
| Up to 3.24MW of protection in a single module | Self-diagnostics for predictive maintenance | 5-year maintenance intervals | Reduced CapEx
| Battery-free flywheel energy storage, Li-Ion or VRLA battery-backed options

PILLER GROUP GmbH | AUSTRALIA | CHINA | FRANCE | GERMANY | INDIA | ITALY | SINGAPORE | SPAIN | UK | USA | PARTNERS WORLDWIDE

A Langley Holdings Company

Issue 45 • August 2022 | 63
 › Zero Downtime

Listen to the
DCD podcast,
new episodes
available every
two weeks.

Hear from
Microsoft, Intel,
Google, Digital
Realty, and more!
Brought to you in
partnership with Vertiv