Introduce Apache OpenDAL (Open Data Access Layer) Provider

[gopidesupavan]

OpenDAL (Open Data Access Layer) is a unified abstraction layer that simplifies interactions with various storage backends, including AWS S3, GCS, Azure, and key-value storage systems. Its guiding theme is: "One Layer, All Storage."

Users can define their task like this.

write_task = OpenDALTaskOperator(
        task_id='example_opendal_write_task',
        action='write',
        data='hello world',
        opendal_config={
            'source_config': {
                'conn_id': 'aws_default',
                'operator_args': {
                    'bucket': 'test-bucket',
                },
                'path': '/test/file/hello.txt'
            },
        },
    )

To Copy from AWS to GCP users can provide two connecton ids, which are airflow connections.

copy_task  = OpenDALTaskOperator(
        task_id='example_opendal_copy_task',
        action='copy',
        opendal_config={
            'source_config': {
                'conn_id': 'aws_default',
                'operator_args': {
                    'bucket': 'aws-bucket,
                },
                'path': '/test/file/hello.txt'
            },
            'destination_config': {
                'conn_id': 'gcp_default',
                'operator_args': {
                    'bucket': 'gcp-bucket',
                },
                'path': '/test/file/hello.txt'
            },
        },
    )

To Copy from GCP to AWS users can provide two connecton ids, which are airflow connections.

copy_task  = OpenDALTaskOperator(
        task_id='example_opendal_copy_task',
        action='copy',
        opendal_config={
             'source_config': {
                'conn_id': 'gcp_default',
                'operator_args': {
                    'bucket': 'gcp-bucket',
                },
                'path': '/test/file/hello.txt'
            },
            'destination_config': {
                'conn_id': 'aws_default',
                'operator_args': {
                    'bucket': 'aws-bucket',
                },
                'path': '/test/file/hello.txt'
            },
        },
    )

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.