Skip to content

File streams handling improvement for oc-azure-storage-adapter #62

Open
@pbazydlo

Description

@pbazydlo

Implementing workaround/hack for the private/public file permissions (described in #61) have had a side effect when putting file content when content is provided as a stream.

After we have written the stream to private container, the stream is at its end. I have not been able to figure out how to reset this stream to beginning for writes to public container, I've tried the following:

  • use fs.read with empty buffer, read length of 0 and position set to 0 in an attempt to reset the stream
    • this had no effect on stream whatsoever
  • pipe through azure write stream
    • as one could expect azure write stream is not pass-through, so any further reads would basically hang

In order to be able to write the actual content to public container after using up original stream with file content, I set up additional stream from azure private container to azure public container. Even though it puts ugly and unnecessary strain on the network I opted in to do this because:

  • I couldn't solve original issue
  • I did not want to load files into memory, that would put unnecessary memory pressure on registry and I think it could potentially block event loop

How to resolve this issue

Figure out a way to either:
a) Solve #61
b) Reset the stream to it's beginning so that it can be reused

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions