Oracle Recovery Manager (RMAN)
Best Practices for Cloud Backups
Tim Chien
Senior Director of Product Management
Oracle Backup & Recovery Technologies
RMAN, Recovery Appliance, Oracle Secure Backup, DB Backup Cloud Service
Copyright © 2019 Oracle and/or its affiliates.
Safe Harbor
The following is intended to outline our general product direction. It is intended for information purposes
only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code,
or functionality, and should not be relied upon in making purchasing decisions. The development,
release, timing, and pricing of any features or functionality described for Oracle’s products may change
and remains at the sole discretion of Oracle Corporation.
Statements in this presentation relating to Oracle’s future plans, expectations, beliefs, intentions and
prospects are “forward-looking statements” and are subject to material risks and uncertainties. A detailed
discussion of these factors and other risks that affect our business is contained in Oracle’s Securities and
Exchange Commission (SEC) filings, including our most recent reports on Form 10-K and Form 10-Q
under the heading “Risk Factors.” These filings are available on the SEC’s website or on Oracle’s website
at http://www.oracle.com/investor. All information in this presentation is current as of September 2019
and Oracle undertakes no duty to update any statement in light of new information or future events.
Copyright © 2019 Oracle and/or its affiliates.
Agenda
• Database Backup Cloud Module for OCI
• Cloud Backup & Recovery Practices
• Migrating Backups to OCI from non-OCI Cloud Platforms
• Archiving Backups for Compliance using Events Service and
Serverless Functions
• Q&A
Copyright © 2019 Oracle and/or its affiliates.
DB Backup Cloud Module for OCI ExaCS
DBCS
DB Backup Cloud Service
• Key based authentication vs. username/password
• Stronger security
• Simplified Management (password changes do not affect backups) On-Premise Databases
• Supports multiple compartments for buckets
• Separation of duties
• Object Lifecycle Policies for archiving
• Lower costs for long-term retention backups
• Multipart upload
• Faster uploads, fewer objects
Copyright © 2019 Oracle and/or its affiliates.
DB Backup Cloud Module Installer
• New oci_install.jar installer available on oracle.com
• https://www.oracle.com/database/technologies/oracle-cloud-backup-downloads.html
• Prepare for installation obtaining:
• Tenancy OCID
• Compartment OCID
• User OCID
• Private key file (corresponding public key must be uploaded via User management console)
• Public key fingerprint
NOTE: Do not use a passphrase with your private key
Copyright © 2019 Oracle and/or its affiliates.
Running the Installer
java -jar oci_install.jar \
-host https://objectstorage.us-ashburn-1.oraclecloud.com \
-pvtKeyFile ~/oci_api_key.pem \
–pubFingerPrint 21:b1:ab:a0:b0:f0:50:30:ee:d6:a7:18:b3:50:a8:36 \
-tOCID ocid1.tenancy.oc1..aaaaaaaaj4ccqe763dizkrcdbssx7ufvlmokd24mb6utvkymyo2xwxyv3gfa \
-cOCID ocid1.compartment.oc1..aaaaaaaaxslr7vtt5cj4ksb3lvwu6agbvo5gh7t5iljd4ydfolgfy4wdpnrq \
-uOCID ocid1.user.oc1..aaaaaaaaid4hi2kzgbbyzjtietoaxxh2gzk4r2bqqqxwag7cqli5cpw6ls4a \
-bucket OCIbucket \
-enableArchiving true \
-archiveAfterBackup 0 days \
-retainAfterRestore 48 hours \
-walletDir ~/ociwallet -libDir ~/ocilib -configfile ~/ociconfig/opcORCL.ora
Copyright © 2019 Oracle and/or its affiliates.
Object Lifecycle Policy Support
If enableArchiving option is set to true, a Lifecycle Policy is applied
to the bucket
This is how it shows up in the Object Storage Cloud Console:
Copyright © 2019 Oracle and/or its affiliates.
RMAN Archive to Cloud Operations
List of Backup Sets
===================
List of Backup Sets
===================
CONFIGURE
-------
CHANNEL
BS Key Type LV Size
BS Key---- Type LV Size
-- ---------- Device
-----------
DEVICE
Type Elapsed
------------
TYPE 'SBT_TAPE' FORMAT '%d_%U' PARMS
Device Type Elapsed Time Completion Time
Time Completion Time
---------------
'SBT_LIBRARY=/home/oracle/ocilib/libopc.so
179
------- Full
---- -- 256.00K SBT_TAPE
---------- ----------- 00:00:02
------------ 14-SEP-19
---------------
ENV=(OPC_PFILE=/home/oracle/opcORCL.ora)';
179BP Key:
Full 179
Handle:
Status:SBT_TAPE
256.00K
89ubntom_1_1
AVAILABLE 00:00:02
Media:
Compressed: YES Tag: TAG20190914T100406
14-SEP-19
objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22
BP Key: 179 Status: AVAILABLE Compressed: YES Tag: TAG20190914T100406
List of Datafiles
Handle: in backup set 179
89ubntom_1_1 Media: objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22
ListLVofType
File Datafiles
Ckp SCNin backup set 179
Ckp Time Abs Fuz SCN Sparse Name
If enableArchiving is true backup pieces are archived automatically by Object Storage
----
File-- LV
----Type
----------
Ckp---------
SCN Ckp -----------
Time------
Abs ----
Fuz SCN Sparse Name
Service (must be granted permission to manage objects)
21---- --Full
using
---- 4715399
channel
14-SEP-19
---------- ---------
ORA_SBT_TAPE_1
21 Full 4715399 14-SEP-19
----------- ------ NO
---- /ade/b/3380669573/oracle/dbs/tbs_25.f
NO /ade/b/3380669573/oracle/dbs/tbs_25.f
using
usingchannel
channelORA_DISK_1
ORA_SBT_TAPE_1
using channel ORA_DISK_1
Archived backups must be restored to Standard Object Storage before RMAN can access
archived logs generated after SCN 4715399 not found in repository
recovery
archivedwill logsbegenerated
done up to SCNSCN
after 4715399
4715399 not found in repository
them for actual DB restore or recovery operations
Media recovery start SCN is
recovery will be done up to SCN 47153994715399
Recovery must bestart
Media recovery doneSCN beyond SCN 4715399 to clear datafile fuzziness
is 4715399
Recovery must be done beyond SCN 4715399 to clear datafile fuzziness
RMAN==========================================================
RESTORE PREVIEW – displays archived backup pieces as “remote”
Initiated recall for the following list of remote backup files
List of remote backup files
RMAN ============================
RESTORE
Handle: 89ubntom_1_1 PREVIEW Media:RECALL c
– initiates restore from archive to standard
objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22 object storage
validation
Handle:succeeded for backupMedia:
89ubntom_1_1 c
piece objectstorage.us-ashburn-..ecloud.com/n/oradbclouducm/tdemoaug22
validation
Finished succeeded
restore for backup piece
at 14-SEP-19
Finished restore at 14-SEP-19
Copyright © 2019 Oracle and/or its affiliates.
RMAN Backup Practices
RMAN> SET ENCRYPTION ON IDENTIFIED BY 'abc123' ONLY;
RMAN> CONFIGURE COMPRESSION ALGORITHM 'MEDIUM';
RMAN>BACKUP DEVICE TYPE SBT AS COMPRESSED BACKUPSET DATABASE PLUS
ARCHIVELOG FORMAT '%d_%U';
RMAN> CONFIGURE DEVICE TYPE 'SBT_TAPE' PARALLELISM 4 BACKUP TYPE TO
BACKUPSET;
RMAN> BACKUP SECTION SIZE 200M TABLESPACE USERS;
Copyright © 2019 Oracle and/or its affiliates.
Backup Pieces to Cloud Objects
• RMAN creates a number of backup pieces using names based
on the FORMAT parameter like ‘%d_%U’
%d -> DBNAME
%U -> system generated unique identifier
For example: ORCL_ctua720h_1_1
• Cloud objects created for this backup piece are:
sbt_catalog/ORCL_ctua720h_1_1/metadata.xml
file_chunk/<DBID>/<DBNAME>/backuppiece/<DATE>/ORCL_ctua720h_1_1/<INCARNATION>/<CHUNK#>
file_chunk/<DBID>/<DBNAME>/backuppiece/<DATE>/ORCL_ctua720h_1_1/<INCARNATION>/metadata.xml
Copyright © 2019 Oracle and/or its affiliates.
Backup Pieces to Cloud Objects
ORCL_ctua720h_1_1
Copyright © 2019 Oracle and/or its affiliates.
RMAN Restore Practices
Daily CROSSCHECK: To ensure that Cloud backup pieces are available for restore.
Monthly RESTORE VALIDATE CHECK LOGICAL: To confirm that a restore can be performed
in the event of a disaster.
Quarterly Full Restore and Recovery: To test DR strategy.
Copyright © 2019 Oracle and/or its affiliates.
Migrating Backups to
OCI from non-OCI Cloud
Platforms
Copyright © 2019 Oracle and/or its affiliates.
Migrating DB Backups to OCI from other Cloud
Platforms
• The object format and naming are the same for:
• OCI native DB Backup Cloud Module
• Legacy Swift-based DB Backup Cloud Module
• OSB Cloud Module for AWS S3
• Backups can be migrated to OCI using tools like rclone
• RMAN ‘catalog backuppiece’ not required
Copyright © 2019 Oracle and/or its affiliates.
rclone example: migrating from AWS S3
• Download rclone (https://rclone.org)
• Prepare your OCI target installing the DB Cloud Backup Module and setting up S3 compatible keys for your user
• Set your environment variables for source and target services
export RCLONE_CONFIG_S3_TYPE=s3
export RCLONE_CONFIG_S3_ACCESS_KEY_ID=AKIRGGSJRV23S5AG4N
export RCLONE_CONFIG_S3_SECRET_ACCESS_KEY=TLJkltRDASlSlhVRPsRuJse2FtWLnFD5
export RCLONE_CONFIG_S3_REGION=us-east-1
export SOURCE=s3:osbbackups
export RCLONE_CONFIG_OCI_TYPE=s3
export RCLONE_CONFIG_OCI_ACCESS_KEY_ID=b8d65742ca7385eac87091f1c0e86376d1e30eb4
export RCLONE_CONFIG_OCI_SECRET_ACCESS_KEY=26TtH1CVKSSFgddsEPwDoBqweDPCsLVrapmerolAsDg=
export RCLONE_CONFIG_OCI_REGION=us-ashburn-1
export RCLONE_CONFIG_OCI_ENDPOINT=https://ixhf9gsbcsml.compat.objectstorage.us-ashburn-1.oraclecloud.com
• rclone --verbose --cache-workers 64 --transfers 64 --retries 32 copy $SOURCE oci:OCIbucket
Copyright © 2019 Oracle and/or its affiliates.
Archiving Backups for
Compliance
Example Using Events Service and
Serverless Functions
Copyright © 2019 Oracle and/or its affiliates.
Compliance Backups
• End-Of-Month or End-Of-Year backups
• Multi-year retention
• Selectively Replicated to Off-Region WORM Buckets (DBAs
have read-only capabilities)
• Policy-based automatic backup deletion in off-region buckets
Copyright © 2019 Oracle and/or its affiliates.
Using Events Service and Serverless Functions
us-ashburn-1 us-phoenix-1
OCIBucket ArchiveBucket
Create Object event
Event Service rule: triggers serverless
sbt_catalog\*MONTHLY* function
file_chunk\*MONTHLY*
RMAN
BACKUP DEVICE TYPE SBT FORMAT ‘MONTHLY_%d_%U’ DATABASE PLUS ARCHIVELOG;
BACKUP FORMAT ‘MONTHLY_CF_%d_%U’ CURRENT CONTROLFILE SPFILE;
NOTE: Events are not guaranteed – use RMAN ‘restore validate’ to verify backups are complete and
recoverable
Copyright © 2019 Oracle and/or its affiliates.
User Privileges
• User belongs to group that has full control on buckets and
objects in us-ashburn-1 region and read-only in us-phoenix-1
region
Allow group BRPM-IAD to manage buckets in compartment brpm where request.region = 'iad‘
Allow group BRPM-IAD to manage objects in compartment brpm where request.region = 'iad‘
Allow group BRPM-IAD to manage objects in compartment brpm where all {request.region = 'phx',
any {request.permission = 'OBJECT_INSPECT', request.permission = 'OBJECT_READ'}}
Allow group BRPM-IAD to manage buckets in compartment brpm where all {request.region = 'phx',
any {request.permission = 'BUCKET_INSPECT', request.permission = 'BUCKET_READ'}}
Copyright © 2019 Oracle and/or its affiliates.
“Archive After 5 Days” Rule on ArchiveBucket
Copyright © 2019 Oracle and/or its affiliates.
5-Year Delete Policy on ArchiveBucket
Copyright © 2019 Oracle and/or its affiliates.
Event Rules
Copyright © 2019 Oracle and/or its affiliates.
Event Code Example
{
"cloudEventsVersion" : "0.1",
"eventID" : "8ba2d00b-b596-4338-b49f-4824baee4677",
"eventType" : "com.oraclecloud.objectstorage.createobject",
"source" : "objectstorage",
"eventTypeVersion" : "1.0",
"eventTime" : "2019-08-21T00:48:41Z",
"schemaURL" : null,
"contentType" : "application/json",
"extensions" : {
"compartmentId" : "ocid1.compartment.oc1..aaaaaaaaxslr7vtt5cj4ksb3lvwu67gbvo5gh7t5iljdmydfolgfygwdpnrq"
},
"data" : {
"compartmentId" : "ocid1.compartment.oc1..aaaaaaaaxslr7vtt5cj4ksb3lvwu67gbvo5gh7t5iljdmydfolgfygwdpnrq",
"compartmentName" : "BRPM",
"resourceName" : "sbt_catalog/MONTHLY_ORCL_1527520098_83u9nk6r_1_1/metadata.xml",
"resourceId" : "",
"availabilityDomain" : null,
"freeFormTags" : { },
"definedTags" : { },
"additionalDetails" : {
"eTag" : "43da49ca-720c-4c96-8b52-175c65a3bfb8",
"namespace" : "oradbclouducm",
"archivalState" : "Available",
"bucketName" : "OCIbucket",
"bucketId" : "ocid1.bucket.oc1.iad.aaaaaaaakfrmfdzueqrrn3nt4gd4ejp4xijycygqzm6heymibpx2iyujqmvq"
}
}
}
Copyright © 2019 Oracle and/or its affiliates.
Serverless Function Code
import io
import json
import oci
import sys
def do(signer,bucket,namesp,object,compartment):
try:
from fdk import response
object_storage_client = oci.object_storage.ObjectStorageClient({},
signer=signer)
def handler(ctx, data: io.BytesIO=None):
response = object_storage_client.copy_object(namesp,bucket,
try:
oci.object_storage.models.CopyObjectDetails(
signer = oci.auth.signers.get_resource_principals_signer()
source_object_name = object,
destination_bucket = 'archivebucket',
# Parse Json to extract variables
destination_region = 'us-phoenix-1',
destination_namespace = namesp,
resp = do(signer,bucketsource,namespace,objectname,compid)
destination_object_name = object )
print("EventType " + str(eventtype) + " " + str(objectname),
)
flush=True, file=sys.stderr)
except (Exception, ValueError) as ex:
print("ERROR: " + str(ex), flush=True, file=sys.stderr)
return response.Response(ctx,
return {"response": str(ex)}
response_data=json.dumps(resp),
return {"response": str(response)}
headers={"Content-Type": "application/json"} )
Copyright © 2019 Oracle and/or its affiliates.
Resources
• tinyurl.com/maacloudpractices
• Best Practices for On-Premise Database Backup & Recovery
• OCI Exadata Backup & Restore Best Practices using Cloud
Object Storage
• Oracle Database Backup Service - FAQ (Doc ID 1640149.1)
• Cloud Backup Performance Analysis (Doc ID 2078576.1)
• Multi-Section Backups (Doc ID 406295.1)
• Master Note For Transparent Data Encryption (TDE)
(Doc ID 1228046.1)
Thank You
