feat: Add support for Trino 451 #609

sbernauer · 2024-07-05T10:03:44Z

Description

Please add a description here. This will become the commit message of the merge request later.

Definition of Done Checklist

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
Please make sure all these things are done and tick the boxes

# Author
- [ ] Changes are OpenShift compatible
- [ ] CRD changes approved
- [ ] CRD documentation for all fields, following the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
- [ ] Helm chart can be installed and deployed operator works
- [ ] Integration tests passed (for non trivial changes)
- [ ] Changes need to be "offline" compatible

# Reviewer
- [ ] Code contains useful comments
- [ ] Code contains useful logging statements
- [ ] (Integration-)Test cases added
- [ ] Documentation added or updated. Follows the [style guide](https://docs.stackable.tech/home/nightly/contributor/docs-style-guide).
- [ ] Changelog updated
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)

# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added
- [ ] [Roadmap](https://github.com/orgs/stackabletech/projects/25/views/1) has been updated

sbernauer · 2024-07-05T11:39:45Z

Started https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/trino-operator-it-custom/122/

maltesander

Nice thanks, LGTM! Looking forward to kick out 414 completely next release :)

sbernauer · 2024-07-05T13:35:23Z

Keycloak failed to install - but it worked in kind...
Started another full (not nightly) test run in https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/trino-operator-it-custom/123/

sbernauer · 2024-07-05T13:43:50Z

The test currently fails with

    logger.go:42: 15:42:21 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | running command: [sh -c kubectl exec -n $NAMESPACE trino-test-opa-0 -- python /tmp/check-opa.py -n $NAMESPACE]
    logger.go:42: 15:42:21 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | /tmp/check-opa.py:448: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    logger.go:42: 15:42:21 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |   timestamp = datetime.utcnow().isoformat(sep=' ', timespec='milliseconds')
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | Traceback (most recent call last):
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |   File "/tmp/check-opa.py", line 484, in <module>
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.407] - admin                -> SHOW CATALOGS
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [['iceberg'], ['lakehouse'], ['system'], ['tpcds'], ['tpch']]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.455] - admin                -> SHOW CATALOGS
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [['iceberg']]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.490] - admin                -> SET SESSION iceberg.test=true
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.514] - admin                -> SHOW SCHEMAS in tpch
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [['information_schema'], ['sf1'], ['sf100'], ['sf1000'], ['sf10000'], ['sf100000'], ['sf300'], ['sf3000'], ['sf30000'], ['tiny']]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.614] - admin                -> SHOW SCHEMAS in system
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [['information_schema'], ['jdbc'], ['metadata'], ['runtime']]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.677] - admin                -> CREATE SCHEMA IF NOT EXISTS iceberg.test WITH (location = 's3a://trino/iceberg/test')
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.699] - admin                -> SHOW CREATE SCHEMA iceberg.test
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [["CREATE SCHEMA iceberg.test\nAUTHORIZATION USER admin\nWITH (\n   location = 's3a://trino/iceberg/test'\n)"]]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.739] - admin                -> ALTER SCHEMA iceberg.test SET AUTHORIZATION admin
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.760] - admin                -> ALTER SCHEMA iceberg.test SET AUTHORIZATION ROLE public
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.775] - admin                -> ALTER SCHEMA iceberg.test RENAME TO test1
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.803] - admin                -> SHOW TABLES in tpch.sf1
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [['customer'], ['lineitem'], ['nation'], ['orders'], ['part'], ['partsupp'], ['region'], ['supplier']]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.865] - admin                -> CREATE TABLE IF NOT EXISTS iceberg.test.test (col1 bigint, col2 bigint)
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.890] - admin                -> COMMENT ON TABLE iceberg.test.test IS 'This is a test table!'
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.916] - admin                -> COMMENT ON COLUMN iceberg.test.test.col1 IS 'This is a column comment!'
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.940] - admin                -> ALTER TABLE iceberg.test.test SET AUTHORIZATION admin
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:21.965] - admin                -> ALTER TABLE iceberg.test.test ADD COLUMN col3 bigint
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:22.005] - admin                -> ALTER TABLE iceberg.test.test RENAME COLUMN col3 TO col_renamed
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:22.049] - admin                -> ALTER TABLE iceberg.test.test DROP COLUMN col_renamed
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | []
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:22.093] - admin                -> DESCRIBE iceberg.test.test
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [['col1', 'bigint', '', 'This is a column comment!'], ['col2', 'bigint', '', '']]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:22.179] - admin                -> INSERT INTO iceberg.test.test VALUES (1,2),(3,4),(5,6)
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [[3]]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [2024-07-05 13:42:22.527] - admin                -> SELECT * FROM iceberg.test.test
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | [[1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 4], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [1, 2], [3, 4], [5, 6], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6], [1, 2], [3, 4], [5, 6]]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |     main()
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |   File "/tmp/check-opa.py", line 480, in main
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |     test.run()
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |   File "/tmp/check-opa.py", line 443, in run
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |     assert result == test["expected"]
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | AssertionError
    logger.go:42: 15:42:22 | opa-authorization_trino-latest-451_hive-3.1.3_opa-0.66.0_keycloak-25.0.0_openshift-false/40-copy-scripts | command terminated with exit code 1

maltesander · 2024-07-08T05:03:12Z

test: https://ci.stackable.tech/view/02%20Operator%20Tests%20(custom)/job/trino-operator-it-custom/124/

razvan

Reviewed on Malte's request. Did not run the tests again.

sbernauer added 2 commits July 5, 2024 12:03

feat: Add support for Trino 451

009a04c

changelog

ae29816

sbernauer self-assigned this Jul 5, 2024

sbernauer added 2 commits July 5, 2024 13:08

fix: JVM config

ab209f6

fix tests

9a8e541

maltesander previously approved these changes Jul 5, 2024

View reviewed changes

sbernauer assigned maltesander and unassigned sbernauer Jul 5, 2024

increase keycloak resources

554e896

maltesander dismissed their stale review via 554e896 July 5, 2024 15:41

fix truncate test

5f25cfc

fix hive examples db user credentials

54b378d

maltesander mentioned this pull request Jul 8, 2024

Update products for 24.7 stackabletech/issues#594

Closed

fix linter

17fc773

razvan approved these changes Jul 8, 2024

View reviewed changes

maltesander added this pull request to the merge queue Jul 8, 2024

Merged via the queue into main with commit 5de8d79 Jul 8, 2024
32 checks passed

maltesander deleted the feat/451 branch July 8, 2024 13:17

maltesander mentioned this pull request Jul 9, 2024

[NEW VERSION] Trino 449-450 #604

Closed

6 tasks

lfrancke added the release/24.7.0 label Jul 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat: Add support for Trino 451 #609

feat: Add support for Trino 451 #609

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

maltesander left a comment

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

maltesander commented Jul 8, 2024

Uh oh!

razvan left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feat: Add support for Trino 451 #609

feat: Add support for Trino 451 #609

Uh oh!

Conversation

sbernauer commented Jul 5, 2024

Description

Definition of Done Checklist

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

maltesander left a comment

Choose a reason for hiding this comment

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

sbernauer commented Jul 5, 2024

Uh oh!

maltesander commented Jul 8, 2024

Uh oh!

razvan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!