Considerations for Data Access in the Lakehouse

Considerations for
Data Access in the
Lakehouse
Zachary Friedman
Product Manager at Immuta

Agenda
Introduction to
Lakehouse Concepts for
Governance
Role-Based Access
Control (RBAC) vs.
Attribute-Based Access
Control (ABAC)
Enterprise-Grade
Authorization in
Databricks SQL Analytics

Data Governance meets the Lakehouse

What is a Lakehouse?
■ Let’s do a (brief) history lesson
■ Late 1980’s: the Data Warehouse
■ Early 2010’s: the Data Lake
■ The roaring 20’s: the Data Lakehouse

Key Features of the Lakehouse
Transaction support
Schema enforcement and
governance
BI support
Separate storage from compute
Support for diverse workloads
Scalable security and access control
management
Additional data governance
capabilities such as auditing and
lineage
Data discovery tools such as data
catalogs
Enterprise-Grade Features
Basic Key Attributes

Key Concepts for Authorization in the Lakehouse
● Role-Based Access Control (RBAC)
● Attribute-Based Access Control (ABAC)
● Enforcement Point

Role-Based Access Control (RBAC)
To manage access to resources, group permissions into roles, and assign those roles to users
■ User-Role relationships
■ Role-Permission relationships

Define a User-Role relationship in Databricks SQL Analytics
■ Manage groups using the Admin Console, Groups API,
or SCIM API
■ Add users to groups and remove them

Define a Role-Permission relationship in Databricks SQL Analytics
■ Define the access that a role grants to a user
■ At a high level this can be implemented in terms of
the is_member() function

Attribute-Based Access Control

Attribute-Based Access Control (ABAC)
Represent fine-grained or dynamic permissions based on who the user is and their relationship to the
resource they want to access.
■ User relationship to the resource can be expressed
as a JOIN on user attributes and values of a resource
column

Access Control Dimensions in SQL Analytics

Access Control Dimensions
A user can access sales data,
but not financial data
A user can access a particular
sales opportunity, or a sales
opportunity matching certain
conditions
Row
Table
A user can access only certain
fields of a record, and we can
mask the values of a column
depending on the user trying
to access
Column

Me
Just now
You’re going to need a
framework to manage all of
these access controls across
your Enterprise.

Requirements for Enterprise-Grade Access Controls
Framework
Individuals can be granted
access to query tables and
views by virtue of:
● membership in a group
(role-based)
● possession of an attribute
(attribute-based)
● request and approval by an
admin
● public access
● individual user selection
● access for a specified
period of time
● access only for a specific
purpose
Individuals can be allowed to see
rows in a dataset based on:
● membership in a group with a
corresponding column value
with that group
● possession of an attribute with
a corresponding column value
with that attribute
● filter based on a time column,
so users are entitled to query
only rows with a specific
recency requirement
Row-level policies
Table-level policies
Different users see different
values in specific columns by
virtue of the above discussed
roles, attributes, and purposes;
examples include:
● Masking a column to NULL
● Masking a column using
hashing
● Masking a column to a
constant string
● Other advanced PETs and
Differential Privacy
Column-level policies

Users who are part of the Active Directory
group called finance are allowed to read
profit loss data.
Provided we’ve kept our groups in sync
between our corporate directory and
Databricks, using either the Admin Console,
Groups API, or SCIM API, then we can solve
this requirement simply with:
GRANT SELECT ON TABLE
accounting.profit_loss_statement
TO finance;
Framework for Managing Table-level Access Controls
Users with the attribute executive are
allowed to read sales data.
This one is a bit more complex. First, we
need to store a (user, name, value) triple
in some sort of attributes table.
Next, we’ll actually need to create a
secure view on top of the original table,
since we can’t pass a WHERE clause as a
principle, only user or group.
ABAC
RBAC

Solving for ABAC in our Framework
Users with the attribute executive are allowed to read sales data.

Restrict the user to only be able to view their own personal attributes.

Putting it all together. Users with the attribute executive are allowed to read sales data.

Managing Row-level Access Controls
A user can access a particular sales opportunity, or a sales opportunity matching certain conditions.
■ Let’s consider a sales dataset that has a territory
column, and we only want users with the attribute
territory to be able to see rows with the
corresponding value in the territory column

fct_sales
sale_id amount territory
1 1000000 US-EAST
2 150000 US-EAST
3 175000 EU
4 800000 APAC
5 50000 US-WEST
6 75000 US-CENTRAL
7 50000 US-EAST

Row-level ABAC
A user can access a particular sales opportunity, or a sales opportunity matching certain conditions.

sec_fct_sales
visible sale_id amount territory
YES 1 1000000 US-EAST
YES 2 150000 US-EAST
NO 3 175000 EU
NO 4 800000 APAC
NO 5 50000 US-WEST
NO 6 75000 US-CENTRAL
YES 7 50000 US-EAST

Column-level Masking
Only executives can see the amount of a sale.

sec_fct_sales (for user without the executive attribute)
visible sale_id amount territory
YES 1 NULL US-EAST
YES 2 NULL US-EAST
NO 3 NULL EU
NO 4 NULL APAC
NO 5 NULL US-WEST
NO 6 NULL US-CENTRAL
YES 7 NULL US-EAST

Thanks for coming to my
talk. My name is Zachary
and I’m a product
manager at Immuta,
which provides an
Enterprise-grade access
controls platform to Data
teams just like this. AMA!
Thank You!

Feedback
Your feedback is important to us.
Don’t forget to rate and review the sessions.

Considerations for Data Access in the Lakehouse

More Related Content

What's hot (20)

Similar to Considerations for Data Access in the Lakehouse (20)

More from Databricks (20)

Recently uploaded (20)

Considerations for Data Access in the Lakehouse