Add support for AWS postgresql standard log format

[kaiyan-sheng]

What does this PR do?

AWS PostgreSQL logs are separated with : instead of the supported , and <space> in https://github.com/elastic/integrations/blob/main/packages/postgresql/data_stream/log/elasticsearch/ingest_pipeline/default.yml#L24-L29. This PR is to create a new pipeline called pipeline-aws-log.yml and use : as separator instead.

AWS PostgreSQL log format

%t:%r:%u@%d:[%p]:t

%t – Time of log entry.
%r – Remote host address.
%u@%d – User name @ database name.
[%p] – Process ID if available.
t - actual log message

Sample log:

2019-03-10 03:54:59 UTC:10.0.0.123(52834):postgres@tstdb:[20175]:ERROR: column "wrong" does not exist at character 8

From the log format you can see, AWS PostgreSQL logs are separated with : between different fields.

Source: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_LogAccess.Concepts.PostgreSQL.html

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Relates Support AWS postgresql log in postgresql integration #3540

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-07-18T21:25:03.741+0000

• Duration: 19 min 50 sec

Test stats 🧪

Test	Results
Failed	0
Passed	59
Skipped	0
Total	59

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (1/1)	💚
Files	100.0% (4/4)	💚 2.92
Classes	100.0% (4/4)	💚 2.92
Methods	93.333% (28/30)	👍 3.788
Lines	97.041% (164/169)	👍 7.176
Conditionals	100.0% (0/0)	💚

[ravikesarwani]

Thanks Kaiyan for driving the fix to this issue.
@rameshelastic/services team let us know if there are any questions. Appreciate your help in getting it reviewed so that we can get this delivered and in the hands of cloud observability team.

[ishleenk17]

Thanks Kaiyan for driving the fix to this issue. @rameshelastic/services team let us know if there are any questions. Appreciate your help in getting it reviewed so that we can get this delivered and in the hands of cloud observability team.

@ravikesarwani : I'll take a look at it

[tommyers-elastic]

looking good - couple of questions though

[girodav]

LGTM :)

[ishleenk17]

Looks good!