11 November 2024, Mälardalen University, AI for Society Seminar

Navigating the White-Water World

with Digital Humanism
Emergent Intelligent Technologies between Utopia and Dystopia

Gordana Dodig Crnkovic

Senior Professor of Computer Science at Mälardalen University and
Professor of Interaction Design, Chalmers University of Technology, Sweden, http://gordana.se/
Navigating the White-Water World
with Digital Humanism
Emergent Digital Technologies between Utopia and Dystopia

Gordana Dodig Crnkovic

Senior Professor of Computer Science at Mälardalen University and
Professor of Interaction Design, Chalmers University of Technology, Sweden, http://gordana.se/
 21 March 2024/ 14:30 -16:30, USI Campus Est
https://www.usi.ch/en/feeds/27126

University of Italian Switzerland, or University of Lugano

Navigating the White-Water World

with Digital Humanism
Emergent Intelligent Technologies between Utopia and Dystopia

Gordana Dodig Crnkovic

Senior Professor of Computer Science at Mälardalen University and
Professor of Interaction Design, Chalmers University of Technology, Sweden, http://gordana.se/
 Proyecto ELAI:
Lecciones éticas de la
inteligencia artificial Carlos III University of Madrid
Ethical Lessons of Artificial Intelligence

Gordana Dodig-Crnković
Mälardalen University &
Chalmers University of Technology,
Sweden

Navigating the White-Water World

with Digital Humanism

Alexandra Koch, Pixabay

April 12th, 2024

https://www.youtube.com/watch?v=Mccpq8fpUI8
Carlos III University of Madrid
https://demaquinaseintenciones.wordpress.com/elai/
Salón de Grados, Edificio Padre Soler, campus de Leganés. https://www.usi.ch/en/feeds/27126 12 April 2024

Navigating the White-Water World

with Digital Humanism
Emergent Intelligent Technologies between Utopia and Dystopia

Gordana Dodig Crnkovic

Senior Professor of Computer Science at Mälardalen University and
Professor of Interaction Design, Chalmers University of Technology, Sweden, http://gordana.se/
 6
https://tiss.tuwien.ac.at/course/courseDetails.xhtml?dswid=5766&dsrid=142&semester=2024S&courseNr=199108
 My affiliations

School of Innovation, Design and Department of Computer Science

Engineering and Engineering

Division of Computer Science and Computer Science and Software

Software Engineering Engineering Division

Research groups: Research groups:

Artificial Intelligence and Intelligent Interaction Design and Software
Systems Engineering
Ubiquitous Computing Critical Robotics

7
My background - from formal to natural languages

PhD in Physics, 1988 PhD in Computing, 2006 Current: Morphological

On Alpha-decay, Department of Computer Science, Computing and Cognition
Physics, University of Zagreb Mälardalen University AI Ethics, Digital Ethics,
Digital Humanism
8
 Transformative emerging intelligent technologies

• We live in an era of transformative AI technologies that profoundly alter our

civilization, reshape existing software and hardware, and challenge our
understanding of fundamental concepts such as intelligence, consciousness,
language, education, research, ethics, sustainability, government, democracy,
being human, and more. It is a process of co-evolution in mutual dependence.

• The pace of technological advancement is accelerating.

• Today's technology isn't an isolated domain managed solely by specialists and

industries. Instead, technology is an integral component of a broader techno-
social system.

• As stakeholders in this development—both professionals and citizens—we

must m a i n t a i n a l o n g - t e r m p e r s p e c t i v e and a c t i v e l y
p a r t i c i p a t e in decision-making about future technologies. We can't
assume that a few years from now technology will remain as it is today.

• The most dramatic development we are experiencing is in AI

(ChatGPT launched in October 2022! GPT in 2019)) 9
 Responses to the current dramatic development
of AI. Examples of collective action

Geoffrey Hinton and Yoshua Bengio warned in

May 2023:

“Mitigating the risk of extinction from AI should

be a global priority alongside other societal-
scale risks such as pandemics and nuclear war,”
MaxTegmark
The letter published by nonprofit organization
Center for AI Safety.

Other signatories include researchers from the

Vector Institute and Mila, as well as professors
from universities across Canada. Open AI CEO
Sam Altman, Microsoft CTO Kevin Scott, etc.
Signatories include: Yoshua Bengio, Stuart
Russell, Gary Marcus, Emad Mostaque, Elon Academics, CEOs sign on in support of AI
regulation and Bill C-27 as Canadian companies
Musk, Tristan Harris, Steve Wozniak and
race to adopt the technology
Yuval Noah Harari.

10
 Recent work on AI regulation

United Nations report (2023)

“Governing AI for Humanity”

https://www.un.org/sites/un2.un.org/files/gov
erning_ai_for_humanity_final_report_en.pdf
UNESCO 2022
‘Recommendation on the Ethics of Artificial Intelligence’
UNESCO's first-ever global standard on AI ethics

UNESCO: The United Nations Educational, Scientific and Cultural Organization

https://www.unesco.org/en/artificial-intelligence/recommendation-ethics
https://www.unesco.org/en/articles/recommendation-ethics-artificial-intelligence
12
USA “AI Bill Of Rights” (2022) - principles

The US AI Bill of Rights outlines principles, including that

people have a r i g h t t o c o n t r o l
how their data is used and
to not be discriminated
against by unfair algorithms.
It is a white paper, which does not have the force of law.
It’s primarily aimed at the
federal
g o v e r n m e n t and could influence
which technologies government
agencies acquire, or help parents, workers,
policymakers, and designers ask tough questions about
artificial intelligence systems.

However, itcan’t constrain large

t e c h c o m p a n i e s , which arguably play a
bigger role in shaping future applications of AI.

https://www.whitehouse.gov/wp-
content/uploads/2022/10/Blueprint-
for-an-AI-Bill-of-Rights.pdf
13
 EU “AI Act” (2024)
The world’s first AI legislation

AI Act, European
Commission. Shaping
Europe’s digital future
https://digital-
strategy.ec.europa.eu/en/policies/regulatory-
framework-ai

https://artificialintelligenceact.eu/ai-act-
explorer/ Source: ISACA

The European Parliament granted final approval of the EU Artificial

Intelligence Act on March 13, 2024, by a vote of 523 for passage, 46
against, and 49 abstaining. The Act faces a final step – approval by EU
member states – as its provisions gradually take effect.
AS S I GNMENT OF RES PONS I BI LI TY:
WHO DECI DES?
Time perspective Stakeholders
- Short-term perspective - Politicians
We, humans, decide - Legislators
- Middle-term perspective - Businesses
AGI & We co-decide - Requirements engineers
- Long-term perspective
- Designers, Developers
Superintelligence? Who decides?
- Programmers
- Deployment engineers, testers
Levels of AI
- Maintenance engineers
- ANI (Narrow AI)
- AGI (Artificial General Intelligence) Learning from experience. Feedback on
development & design
- ASI (Artificial Super Intelligence)

https://tinyurl.com/pjbdyn95 Global AI governance: barriers and pathways forward- Huw Roberts, Emmie Hine,
Mariarosaria Taddeo, Luciano Floridi

15
 Our ’White Water World’ – complex & dynamic

“We are forcing the past as a solution set. But Complexity science gives us a new lens
the past as a solution set is not a viable option. through which to view the world as one that is
We need a new toolset.” entangled and emerging.

Design Unbound presents a new tool set for

having agency in the world today, which we
characterize as a ’white water world’ – one
that is rapidly changing, hyperconnected and
radically contingent.

Imagination is a ’muscle that must

be exercised’ (John Seely Brown)

’Wicked problems’: As soon as you start

Hyperconnectivity causes a transition from
to solve them, they morph.
equilibrium to constant non-equilibrium.
“Computational irreducibility”: You must
The need for adaptivity, anticipation, and
run the model to see the outcome.
resilience.
Computation takes the same time as the
process itself.
16
 AI UTOPI A VS . DYS TOPI A

Utopian Scenarios Dystopian Scenarios

● Mass Unemployment and Economic

● End of Poverty and Scarcity
Inequality
● Universal Access to Knowledge and
● Mass Surveillance and Loss of
Education
Privacy
● Improved Healthcare and Longevity
● AI-Driven Authoritarianism
● Reduction of Human Labor and
● Weaponization of AI
Increased Creativity
● Existential Risk (AI Overreach)
● Personal AI assistants (agents)
● Erosion of Human Autonomy and
● Global Cooperation and Problem-
Solving Agency

17
 CENTRAL OPEN QUESTIONS
• How does the future (may) look like? We know future will not be
like present. [Compare to Egypt. Ancient Egypt: Old Kingdom 500 years, "Age of the
Pyramids” , The Middle Kingdom (c. 2055–1650 BCE) 400 years golden age, The New Kingdom
(c. 1550–1070 BCE) 450 years of the most powerful period.]

• How can we navigate to evade dystopic scenarios in the

turbulent currents of today's world, filled with disruptive
intelligent technologies? (AI controlling humans, taking over,
and eventually destroying humans. Humans with the help of AI
enslaving other humans.)

• How can we envision the broader landscape of a future human-

centered digital society? What would human flourishing mean?

• What does a desirable future look like for both humans and our
planet, steering towards common preferred futures/utopias?

18
 Plan of the talk

● Navigating Possible Futures: Speculative Design

● Complexity & Systemic Thinking

● A White Water World & Emergence

in Ecologies of Change

● Value-based Human-centric Design

● Digital Humanism

● A Case Study: Ethics Of Autonomous Cars

● Wrap-up

19
 We are discussing possible futures
with socially disruptive technologies

Formal sciences: axiomatic systems.

NARS (Non-Axiomatic Reasoning System), an adaptive AI
framework that’s designed to operate under conditions of
uncertain, incomplete information. The combination
allows AI to make decisions based on probability and
evidence rather than deterministic rules, which is essential
for real-world applications where data may be messy or
limited. 20
 Design for possible & preferable futures –
SPECULATIVE DESIGN

Speculative design combines informed, hypothetical extrapolations of

an emerging technology’s development with a deep consideration of
the cultural landscape into which it might be deployed, to speculate on
future products, systems and services.

These speculations are then used to examine and encourage dialogue

on the impact a specific technology may have on our everyday lives.

Auger Loizeau

21
https://elviavasconcelosblog.wordpress.com/2017/01/15/what-is-speculative-critical-fiction-design-part-1/ 21
 Speculative Everything – Antony Dunne and Fiona Raby

Table of Contents:
Beyond radical design?
A map of unreality
Design as critique
Consuming monsters: big, perfect, infectious
A methodological playground: fictional worlds and
thought experiments
Physical fictions: invitations to make believe
"what if" questions Aesthetics of unreality
Between reality and the impossible
https://www.youtube.com/watch?v=kmibm20UsoA Speculative everything. 22
Speculative Design creates space to…

Arrange emerging (not yet

available) technological
‘elements’ to hypothesize
future, products and artifacts.

Apply alternative plans,

motivations, or ideas to those
currently driving technological
development, in order to
facilitate new arrangements of
existing elements.

Develop new perspectives on

big systems.

23
Speculative Design Facilitates…

Exploration of ‘What is a better future

(with respect to the present)?’

Generating a better understanding of

the potential implications of a specific
(disruptive) technology in various
contexts and on multiple scales – with
a particular focus on everyday life.

Moving design ‘upstream’ – to not

simply package technology at the end
Giovanni M Troiano, Matthew Wood, Mustafa
of the technological journey but to Feyyaz Sonbudak, Riddhi Chandan Padte, and
impact and influence that journey Casper Harteveld. 2021. “Are We Now Post-
from its genesis. COVID?”: Exploring Post-COVID Futures Through
a Gamified Story Completion Method. In
Proceedings of the 2021 ACM Designing Interactive
Systems Conference (DIS '21). ACM, New York,
NY, USA, 48–63.
https://doi.org/10.1145/3461778.3462069
24
Speculative Design and its context

https://speculativeedu.eu/new-reflections-on-speculativity/ 25
Complexity & systemic thinking in hyper-connected society

https://humanparts.medium.com/making-sense-of-complexity-ee78755d56b9 Schizophrenia Protein-Protein Interactome

ADD TO THIS PICTURE (INTELLIGENT) INTERNET OF THINGS! 26

 Design Unbound. Designing for emergence in a ’white water world’.
(1) Designing for Emergence & (2) Ecologies of Change

Design Unbound. Designing for

Emergence in a White Water World.

Ann Pendleton-Jullian and

John Seely Brown, MIT Press 2018

https://www.desunbound.com/
https://www.youtube.com/watch?v=-U8h4wNBfCQ
https://www.youtube.com/watch?v=tFPvK1mO6Sg
https://www.youtube.com/watch?v=Lto8szGvPfM
https://www.desunbound.com/assets/DesUnbound_chapter_8.pdf

Richard Buchanan (1992) Wicked Problems in Design Thinking. Design

Issues, Vol. 8, No. 2, pp. 5-21. The MIT Press
http://www.jstor.org/stable/1511637.
27
 THE WAY WE MAKE DECISIONS

VALUE-BASED HUMAN-CENTRIC DESIGN

Values
Values serve as a guide to action
and knowledge.

They are relevant to all aspects of

scientific and engineering practice,
including discovery, analysis, and
application.

TUANA. COMMUNICATIONS OF THE ACM | DECEMBER 2015 | VOL. 58 | NO. 12

28
 A VALUE-BASED DESIGN APPROACH

One question we can ask is: How much time can we afford to spend on the “ideation phase”
before starting to actually build technology?
Andrew Ng points out for a startup it is more profitable to identify which technology can be
built, and then go and build it, instead of spending a lot of time thinking about all possible
alternatives: https://www.youtube.com/watch?v=5p248yoa3oE (29:08)
29
29
 Human-centered future intelligent society

“In the Fourth Industrial Revolution, the convergence of artificial intelligence, robot technology, big
data and software disrupts fields such as labor, welfare, employment, education and defense.
This has sparked revolutionary change across society.”
Wikipedia, https://en.wikipedia.org/wiki/Intelligent_information_society

30
Humanism - Dictionary definition

Humanism /ˈhjuːmənɪz(ə)m/ noun

1.a rationalist outlook or system of thought attaching prime

importance to human rather than divine or supernatural matters.

• a Renaissance cultural movement which turned away from

medieval scholasticism and revived interest in ancient Greek and
Roman thought.

• a system of thought criticized as being centred on the notion

of the rational, autonomous self
and ignoring the conditioned nature of the individual.

Oxford dictionary

31
 Humanism and Nature

Humanism, as a philosophical and ethical stance, has a growing and increasingly

important relationship with the environment and nature, particularly in the context
of environmental humanism.

● Respect for Nature and Interdependence

● Ethics of Sustainability
● Environmental Responsibility as Part of Human Flourishing
● Promotion of Science and Evidence-Based Solutions
● Intrinsic Value of Nature
● Humanism and Climate Change
● A Global, Inclusive Perspective

32
32
 The Digital Humanism Initiative

The Digital Humanism Initiative is an international collaboration seeking to build a

community of scholars, policy makers, and industrial players who are focused on
ensuring that technology development remains centered on human interests.

● Digital humanism is a global, international issue. Humankind is at the center.

● The approach: scientific, transdisciplinary, interdisciplinary, multidisciplinary,
in the tradition of the Enlightenment.
● People are the central focus, as individuals and societies. [this is not in conflict with
environment and animal rights!]*Human flourishing is fundamentally based on human connections with nature
● Technology is for people and not the other way around.
● Building a just and democratic society with humans at the center of
technological progress.

https://dighum.ec.tuwien.ac.at/ Digital Humanism movement web page @ TUW –

Technical University in Vienna
E. Prem, L. Hardman, H. Werthner, P. Timmers (eds.). Research, innovation, and education roadmap for digital
humanism. The Digital Humanism Initiative. Vienna, 2022. https://dighum.ec.tuwien.ac.at/
33
*Mark Coeckelbergh (2024) What is Digital Humanism? Journal of Responsible Technology 17 100073 33
Perspectives on Digital Humanism – Open Access

Hannes Werthner, Erich Prem, Edward A. Lee, and

Carlo Ghezzi (eds): Perspectives on Digital Humanism,
Springer, 2022.
https://link.springer.com/book/10.1007/978-3-030-
86144-5

34
Introduction to Digital Humanism – A Textbook
Open Access

Hannes Werthner, Carlo Ghezzi, Jeff Kramer, Julian

Nida-Rümelin, Bashar Nuseibeh, Erich Prem, and
Allison Stanger (eds): Introduction to Digital
Humanism, Springer, 2024.
https://link.springer.com/book/10.1007/978-3-030-
86144-5

35
Digital Humanism Lecture Series

https://dighum.ec.tuwien.ac.at/news-events/

https://www.youtube.com/@DigitalHumanism Youtube channel

(Stuart Russel, Gary Marcus, Edward Lee, Deborah G. Johnson, Julian Nida-Rümelin,… )

Digital Humanism Manifesto

“Today, we experience the co-evolution of technology and humankind.
The flood of data, algorithms, and computational power is disrupting the very
fabric of society by changing human interactions, societal institutions,
economies, and political structures.
Science and the humanities are not exempt. This disruption simultaneously
creates and threatens jobs, produces and destroys wealth,
and improves and damages our ecology. It shifts power structures, thereby
blurring the human and the machine.”

https://dighum.ec.tuwien.ac.at/dighum-manifesto/ 36
Viable Initiatives in a Hyperconnected, Dynamic, Emergent World

https://medium.com/@bill.bannear/the-new-zeitgeist-relationships-and-emergence-e8359b934e0
37
UNESCO Chair on Digital Humanism
Peter Knees Chair and Julia Neidhardt Co-Chair

“UNESCO uses
education, science,
culture, communication
and information to
foster mutual
understanding and
respect for our planet.”

CAIML - Center for Artificial Intelligence and Machine Learning. https://www.tuwien.at/caiml/

https://informatics.tuwien.ac.at/stories/2383

UNESCO - the United Nations Educational, Scientific and Cultural Organization. 38

 Case study - Autonomous Cars Ethics

Autonomous cars Book chapter:

“Steps Towards Real-world Ethics for Self-driving Cars: Beyond the Trolley
As a special case of intelligent Problem”.
emerging technology Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2021). In Steven John Thompson
(Ed.), Machine Law, Ethics, and Morality in the Age of Artificial Intelligence. IGI
Global

Picture: https://www.aarete.com/insights/what-is-the-business-case-for-autonomous-vehicles-in-the-supply-
chain/

39
 Challenges
• Hardware and software adequacy
• Vulnerabilities of machine-learning algorithms
• Control of trade-offs between safety and other factors
(like economic) in the design, manufacturing and
operation
• Possibility of intervention in case of major failure of
the system and graceful degradation
• Systemic solutions to guarantee safety in
organizations (regulations, authorities, safety culture)

Safety Approaches
• Setting safety as the first priority
• Learning from the history of automation
• Learning from experience of current use
• Specification of how a system will behave in cases
when autonomous operation is disabled (safe mode)
• Preparedness for handling “loss of control”
situations- autonomous systems running amok
• Regulations, guidelines, standards being developed
as the technology develops

40
 Challenges
• Minimal necessary security requirements for
deployment of the system
• Security in the context and connections
• Deployment of software updates
• Storing and using received and generated data
Security in a secure way
Approaches
• Technical solutions to guarantee minimum
security under all foreseeable circumstances
• Anticipation and prevention of the worst-case
scenarios
• Accessibility of data, even in the case of
accidents, learning from experience

41
 Challenges
• Risk of technology causing harm, physical,
cognitive, psychological, social, etc.
• Disruptive changes in the labor market
• Transformation of related businesses, markets,
and business models (manufacturers, insurance,
Non- etc.)
• Loss of human skills
maleficence • Loss of autonomy
& Approaches
Beneficence • Partly covered by technical solutions, but
interdisciplinary approaches are needed
• Preparation of strategic solutions for people
losing jobs
• Learning from historic parallels to
industrialization and automatization
42
 Challenges
• Assignment and distribution of responsibility
Responsibility and accountability as some of central regulative
mechanisms for the development of new
and technology

Accountability Approaches
• The Accountability, Responsibility, and
Transparency (ART) principle (Virginia Dignum)
based on a Design for Values approach that
includes human values and ethical principles in
the design processes

43
 Humans in the loop
Freedom of choice
To what extent will the user be in control?

Will the AI do, what I want it to do?

Stakeholders Implementation of restrictions
Interests Loss of jobs compensation
Impacts on society as a whole

44
 Challenges
• Establishing trust between humans and robots
as well as within the social system involving
Social Trust robots
Approaches
• Further research on how to implement trust
across multiple systems
• Provision of trusted connections between
components as well as external services

45
Value-based Ethical
Guidelines
for Self-Driving Cars

46
 Presented at major SE conference ICSE2020 as poster
Ehics of Self- Extended version in a book chapter:
Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2021). Steps
Driving Cars Towards Real-world Ethics for Self-driving Cars: Beyond the Trolley
Problem. In Steven John Thompson (Ed.), Machine Law, Ethics, and
Morality in the Age of Artificial Intelligence. IGI Global

47
 Our Future with AI

AI FOR A BETTER
FUTURE
An Ecosystem Perspective
on the Ethics of AI
and Emerging Digital Technologies

Bernd Carsten Stahl

https://link.springer.com/book/10.1007/978-3-030-69978-9 OPEN ACCESS

48
 Organizational Ethical Issues of AI

Bernd Carsten Stahl (2021) Artificial Intelligence for a Better Future, An Ecosystem Perspective on the Ethics
of AI and Emerging Digital Technologies https://link.springer.com/book/10.1007%2F978-3-030-69978-9

49
 Overview of AI stakeholders

Bernd Carsten Stahl (2021) Artificial Intelligence for a Better Future, https://link.springer.com/book/10.1007%2F978-3-030-69978-9

50
Key Challenges of Ethical Governance of AI

Bernd Carsten Stahl (2021) Artificial Intelligence for a Better Future, https://link.springer.com/book/10.1007%2F978-3-030-69978-9 51
Practical Use of the Proposed Ethical Program
for Intelligent Emergent Technologies -
Importance of Transdisciplinarity and Transversal Knowledge

Ethical requirements must be fulfilled in all

phases in the life-cycle of technology, in
the context of:
● Conceptualization/Design/Prototyping/
Construction/Development/Testing/Production
● Deployment/Application/

● Maintenance/Support
● Oversight/Regulation
Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2021). In
Steven John Thompson (Ed.), Machine Law, Ethics, and Morality
in the Age of Artificial Intelligence. IGI Global

https://www.indiamart.com/kaynes-tecnology-india-pvt-ltd/product-life-cycle-management.html
52
 Challenges for Emergent Technologies

Legislation Global framework Guidelines Implementation

Keeping Creating and Defining the Including ethical

legislation up-to- defining global guidelines that guidelines in
date with current legislation will be adopted design and
level of frameworks for by society for development
automated the building self- processes
driving, and implementation driving cars
emergence of of interoperable
self-driving cars and
development of
increasingly
automated
vehicles

Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2021). In Steven John Thompson (Ed.), Machine Law, Ethics, and Morality in the Age
of Artificial Intelligence. IGI Global

53
 Building Ethical Technology in an Ethical Way

Work on the s h a r e d v i s i o n o f e m e r g e n t t e c h n o l o g i e s .
A n t i c i p a t i o n and consideration of uncertainties/Speculative design

A s y s t e m - l e v e l a p p r o a c h involving the entire software-hardware system as

well as human stakeholders, with organizational, and social factors.

Multi-criteria decisions. Multidisciplinary approach.

L e a r n i n g f r o m e x p e r i e n c e from the whole life cycle of technology.

Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2021). In Steven John Thompson (Ed.), Machine Law, Ethics, and Morality in the Age
of Artificial Intelligence. IGI Global

54
 Ethical Lessons of Artificial Intelligence
R e s p o n s i b i l i t y i n A I D e v e l o p m e n t : recognizing the responsibility of developers and
engineers to create AI systems that are not only effective but also fair, transparent, and non-discriminatory.

I m p a c t o n S o c i e t y : There are lessons to be learned regarding the societal impact

of AI, such as the potential for job displacement, privacy concerns, and changes in social dynamics.

B i a s a n d F a i r n e s s : AI can inadvertently perpetuate or amplify existing biases if not carefully designed

and monitored. Understanding and addressing these issues is a crucial ethical lesson.

R e s p o n s i b i l i t y i n A I D e v e l o p m e n t : recognizing the responsibility of developers and engineers

to create AI systems that are not only effective but also fair, transparent, and non-discriminatory.

T r a n s p a r e n c y a n d E x p l a i n a b i l i t y : As AI systems become more complex, ensuring that they are

transparent, and their decisions can be explained and understood by humans is an important ethical
consideration.

A c c o u n t a b i l i t y : Establishing clear lines of accountability for AI's decisions and actions, particularly when
they lead to harm or injustice, is an ethical challenge that must be addressed.

S a f e t y a n d S e c u r i t y : Ensuring that AI systems are safe from malicious uses and are secure
against potential breaches is an ongoing ethical concern.

R e g u l a t i o n a n d G o v e r n a n c e : Determining the appropriate level of regulation and the governance

structures needed to oversee AI development and implementation is an essential ethical lesson.

B e n e f i c e n c e a n d N o n m a l e f i c e n c e : AI should be designed and used in ways that benefit people

and society at large while avoiding harm, reflecting these core ethical principles.
55
 Wrap-up
The main topics we visited during this talk

● Navigating Possible Futures: Speculative Design

● A White Water World & Emergence in Ecologies of Change

● Value-based Human-centric Design

● Digital Humanism

● Case Study: Ethics of Autonomous Cars

As AI technology becomes more and more powerful, the old wisdom applies:
“ W i t h g r e a t p o w e r c o m e s g r e a t r e s p o n s i b i l i t y. ”

The perspective of Digital Humanism was presented as a way of approaching the

contemporary white-water world, driven by the prospect of a more humane and
inclusive future based on care for humans and our planet. With human rights,
democracy, inclusion, diversity, care for other living beings and the environment.

56
Q & A TIME!

57
References
● G. Dodig-Crnkovic, T. Holstein, P. Pelliccione and, Jathoosh Thavarasa (2023) "Future Intelligent Autonomous Robots, Ethical by
Design. Lessons Learned from Autonomous Cars Ethics." Proc. ICSIT 2023 conference. ISSN: 2771-6368 (Print) ISBN: 978-1-
950492-70-1 (Print) DOI: 10.54808/ICSIT2023.01 https://www.iiis.org/CDs2023/CD2023Spring//
● Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2021). Steps Towards Real-world Ethics for Self-driving Cars: Beyond the Trolley
Problem. In Steven John Thompson (Ed.), Machine Law, Ethics, and Morality in the Age of Artificial Intelligence. IGI Global
● Holstein, T., Dodig-Crnkovic, G., & Pelliccione, P. (2020). Real-world Ethics for Self-Driving Cars. In Proceedings of the 42nd
International Conference on Software Engineering (ICSE ’20) Poster Track. https://ethics.se
● Holstein, T. Dodig-Crnkovic G. Avoiding the Intrinsic Unfairness of the Trolley Problem. Accepted for the Proceedings of
FairWare workshop at ICSE2018, to be published by ACM.
● Holstein, T. Dodig-Crnkovic G. and Pelliccione P. Ethical and Social Aspects of Self-Driving Cars, http://arxiv.org/abs/1802.04103
● Dodig Crnkovic, G. and B. Çürüklü. Robots: ethical by design. Ethics and Information Technology, 14(1):61–71, Mar 2012.
● Dodig Crnkovic, G. and B. Çürüklü. Robots: ethical by design. Ethics and Information Technology, 14(1):61–71, Mar 2012.
● Dodig-Crnkovic, G. and D. Persson. Sharing moral responsibility with robots: A pragmatic approach. In Proceedings of the 2008
Conference on Tenth Scandinavian Conference on Artificial Intelligence: SCAI 2008, pages 165–168, Amsterdam, The
Netherlands, IOS Press. 2008.
● Dodig-Crnkovic, G. and D. Persson. Sharing moral responsibility with robots: A pragmatic approach. In Proceedings of the 2008
Conference on Tenth Scandinavian Conference on Artificial Intelligence: SCAI 2008, pages 165–168, Amsterdam, The
Netherlands, IOS Press. 2008.
● Johnsen A., G. Dodig- Crnkovic, K. Lundqvist, K. Hänninen, and P. Pettersson. Risk- based decision-making fallacies: Why present
functional safety standards are not enough. In 2017 IEEE International Conference on Software Architecture Workshops (ICSAW),
pages 153–160, April 2017.
● Sapienza, G., Dodig-Crnkovic, G. and I. Crnkovic. Inclusion of ethical aspects in multi-criteria decision analysis. In 2016 1st
International Workshop on Decision Making in Software ARCHitecture (MARCH), pages 1–8, April 2016.
● Thekkilakattil A. and G. Dodig-Crnkovic. Ethics aspects of embedded and cyber-physical systems. In 2015 IEEE 39th Annual
Computer Software and Applications Conference, volume 2, pages 39–44, July 2015.
● Margarita Georgieva (student) and Gordana Dodig-Crnkovic (2011) Who Will Have Irresponsible, Untrustworthy, Immoral
Intelligent Robot? Proceedings of IACAP 2011. The Computational Turn: Past, Presents, Futures?, p 129, Mv-Wissenschaft,
Münster, Århus University, Danmark, Ed. Ess and Hagengruber, July 201

58
● Regulation (E.U.) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural
persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive
95/46/EC (General Data Protection Regulation), 2016.
https://eur-lex.europa.eu/eli/reg/2016/679/oj
● euRobotics Topics Group ‘Ethical Legal and Socio-Economic Issues.’ Policy Documents & Institutions - ethical, legal, and
socio-economic issues of Robotics and artificial intelligence, 2022. https://www.pt-ai.org/TG-ELS/
● F. Operto, “Ethics in Advanced Robotics,” IEEE Robot. Autom. Mag., vol. 18, no. 1, pp. 72–78, Mar. 2011.
● N. Leveson, “Are You Sure Your Software Will Not Kill Anyone?,” Commun. ACM, vol. 63, no. 2, pp. 25–28, Jan. 2020.
https://dl.acm.org/doi/10.1145/3376127
● P. Lin, K. Abney, and G. A. Bekey, Robot Ethics: The Ethical and Social Implications of Robotics. MIT Press, 2011.
http://kryten.mm.rpi.edu/Divine-Command_Roboethics_Bringsjord_Taylor.pdf
● P. M. Asaro, “What should we want from a robot ethic?,” in Machine Ethics and Robot Ethics, 2017.
https://peterasaro.org/writing/Asaro%20IRIE.pdf
● P. M. Asaro, Autonomous Weapons and the Ethics of Artificial Intelligence,” in S. Matthew Liao (ed.) Ethics of Artificial
Intelligence, Oxford University Press, pp. 212-236.
https://global.oup.com/academic/search?q=ethics+of+artificial+intelligence&cc=us&lang=en
● S. G. Tzafestas, Roboethics - A Navigating Overview, vol. 79. Springer International Publishing, 2016.
https://link.springer.com/book/10.1007/978-3-319-21714-7
● V. C. Müller, “Ethics of Artificial Intelligence and Robotics,” in The Stanford Encyclopedia of Philosophy (Summer 2021
Edition), Edward N. Zalta (ed.), URL = https://plato.stanford.edu/archives/sum2021/entries/ethics-ai/
● W. Wallach and C. Allen, Moral Machines: Teaching Robots Right from Wrong. New York: Oxford University Press, 2009.
https://academic.oup.com/book/10768
● https://www.ethics.se ETHICS & SELF-DRIVING CARS
● Baran Çürüklü, Gordana Dodig-Crnkovic, Batu Akan (2010) Towards Industrial Robots with Human Like Moral
Responsibilities, 5th ACM/IEEE International Conference on Human-Robot Interaction, Osaka, Japan, March 2010
● Gordana Dodig-Crnkovic (2010) Information Ethics for Robotic Agents European Computing and Philosophy Conference
ECAP 2010 @The Technische Universität München, 4-6 October, 2010
● Gordana Dodig-Crnkovic (2009) Delegating Responsibilities to Intelligent Robots. ICRA2009 IEEE International
Conference on Robotics and Automation. Workshop on Roboethics Kobe, Japan, May 17, 2009.
● Gordana Dodig-Crnkovic and Daniel Persson (student) (2008) Sharing Moral Responsibility with Robots: A Pragmatic
Approach. Tenth Scandinavian Conference on Artificial Intelligence, SCAI 2008.
Volume 173, Frontiers in Artificial Intelligence and Applications. Eds. A. Holst, P. Kreuger and P. Funk
● Gordana Dodig-Crnkovic and Daniel Persson (student) (2008) Towards Trustworthy Intelligent Robots, NA-CAP@IU 2008,
North American Computing and Philosophy Conference, Indiana University, Bloomington, July 10-12, 200

59
Digital Humanism References

https://www.youtube.com/watch?v=V-XvfMEZgPc The Challenge of Being Humanely Digital - UCAI '22

Keynote by Erich Prem

https://informatics.tuwien.ac.at/digital-humanism/

https://dighum.ec.tuwien.ac.at

https://link.springer.com/book/10.1007/978-3-030-86144-5 Perspectives on Digital Humanism – book

freely available for download

https://dighum.ec.tuwien.ac.at/dighum-manifesto/ Vienna Manifesto on Digital Humanism

https://nextconf.eu/2017/11/what-is-digital-humanism/#gref

https://www.erichprem.at/publications-press-videos/ Erich Prem videos

60