Skip to content

[Kernel] [CatalogManaged] Better class/method docs for TableManager, ResolvedTableBuilder, ResolvedTable #4822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,46 @@
import java.util.List;
import java.util.Optional;

/** Represents a Delta table resolved to a specific version. */
/**
* Represents a Delta table resolved to a specific version.
*
* <p>A {@code ResolvedTable} is a snapshot of a Delta table at a specific point in time, identified
* by a version number. It provides access to the table's metadata, schema, and capabilities for
* both reading and writing data. This interface is the result of resolving a table through a {@link
* ResolvedTableBuilder} and serves as the entry point for table operations.
*
* <p>The resolved table represents a consistent view of the table at the resolved version. All
* operations on this table will see the same data and metadata, ensuring consistency across reads
* and writes.
*/
@Experimental
public interface ResolvedTable {

/** @return the file system path to the this table */
String getPath();

/** @return the version number of this table snapshot */
long getVersion();

/** @return the timestamp (in milliseconds since Unix epoch) of when this version was committed */
long getTimestamp();

/**
* @return the partition columns of the table at this version, in the order they are defined in
* the table schema. Returns an empty list if the table is not partitioned
*/
List<Column> getPartitionColumns();

/**
* @param domain the domain name to query
* @return the configuration for the provided {@code domain} if it exists in the table, or empty
* if the {@code domain} is not present in the table.
*/
Optional<String> getDomainMetadata(String domain);

/** @return the schema of the table at this version */
StructType getSchema();

/** @return a scan builder for constructing scans to read data from this table */
ScanBuilder getScanBuilder();
}
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,50 @@
*/
@Experimental
public interface ResolvedTableBuilder {
/**
* Configures the builder to resolve the table at a specific version.
*
* @param version the version number to resolve to
* @return a new builder instance configured for the specified version
*/
ResolvedTableBuilder atVersion(long version);

// TODO: atTimestamp

/**
* Provides parsed log data to optimize table resolution.
*
* <p>When log data is provided, Kernel can avoid reading from the filesystem for information that
* is already available in the parsed data, improving performance.
*
* @param logData the parsed log data to use for optimization
* @return a new builder instance with the provided log data
*/
/** For now, only log datas of type {@link ParsedLogType#RATIFIED_STAGED_COMMIT}s are supported */
ResolvedTableBuilder withLogData(List<ParsedLogData> logData);

// TODO: P & M must be public interfaces, not internal classes
/**
* Provides protocol and metadata information to optimize table resolution.
*
* <p>When protocol and metadata are provided, Kernel can avoid reading this information from the
* filesystem, improving performance.
*
* @param protocol the protocol information
* @param metadata the metadata information
* @return a new builder instance with the provided protocol and metadata
*/
// TODO: [delta-io/delta#4820] Public Protocol API
// TODO: [delta-io/delta#4821] Public Metadata API
ResolvedTableBuilder withProtocolAndMetadata(Protocol protocol, Metadata metadata);

/**
* Constructs the {@link ResolvedTable} using the provided engine.
*
* <p>This method will read any missing information from the filesystem using the provided engine
* to complete the table resolution process.
*
* @param engine the engine to use for filesystem operations
* @return the resolved table instance
*/
ResolvedTable build(Engine engine);
}
11 changes: 11 additions & 0 deletions kernel/kernel-api/src/main/java/io/delta/kernel/TableManager.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,17 @@ public interface TableManager {
// TODO static ResolvedTable forPathAtTimestamp(Engine engine, String path, long timestamp);

// TODO: Take in a Committer for write support.

/**
* Creates a builder for loading a Delta table at the given path.
*
* <p>The returned builder can be configured to load the table at a specific version or with
* additional metadata to optimize the loading process.
*
* @param path the file system path to the Delta table
* @return a {@link ResolvedTableBuilder} that can be used to load a {@link ResolvedTable} at the
* given path
*/
static ResolvedTableBuilder loadTable(String path) {
return new ResolvedTableBuilderImpl(path);
}
Expand Down
Loading