Skip to content

HIVE-20889: Support timestamp-micros in AvroSerDe #5779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

HIVE-20889: Support timestamp-micros in AvroSerDe #5779

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ce3758f
HIVE-20889: Support timestamp-micros in AvroSerDe
Mar 5, 2025
3602cfc
HIVE-20889: Add microsecond precision methods
May 11, 2025
8fdeb5a
HIVE-20889: Decouple avro schema timestamp check and add toEpochMicro…
May 11, 2025
578f94a
HIVE-20889: extend avro test cases
May 12, 2025
dba1d0d
May 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions common/src/java/org/apache/hadoop/hive/common/type/Timestamp.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,22 @@ public long toEpochMilli() {
return localDateTime.toInstant(ZoneOffset.UTC).toEpochMilli();
}

public long toEpochMicro() {
return localDateTime.toEpochSecond(ZoneOffset.UTC) * 1_000_000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use similar semantics as toEpochMilli()?

return localDateTime.toInstant(ZoneOffset.UTC).toEpochMilli() * 1000 + localDateTime.getNano() / 1000;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing so might lead to loss of precision due to integer overflow. Hence we are parsing the value to seconds here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

localDateTime.toInstant(ZoneOffset.UTC).toEpochMilli() returns a long value and due to addition of all these values the implicit datatype is long hence there should be no chance of integer overflow.

+ localDateTime.getNano() / 1000;
}

public long toEpochMilli(ZoneId id) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't zone conversion also applicable for micros?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not find any code changes related to zone, and hence did not add extra unnecessary changes. Although, adding a similar method for future does make sense.

return localDateTime.atZone(id).toInstant().toEpochMilli();
}

public long toEpochMicro(ZoneId id) {
return localDateTime.atZone(id)
.toInstant()
.getEpochSecond() * 1_000_000L +
localDateTime.getNano() / 1000;
}

public void setTimeInMillis(long epochMilli) {
localDateTime = LocalDateTime.ofInstant(
Instant.ofEpochMilli(epochMilli), ZoneOffset.UTC);
Expand Down Expand Up @@ -236,6 +248,18 @@ public static Timestamp ofEpochMilli(long epochMilli, int nanos) {
.withNano(nanos));
}

public static Timestamp ofEpochMicro(long epochMicro) {
int nanos = (int) ((epochMicro % 1000000) * 1000);
epochMicro -= nanos / 1_000_000;

Instant instant = Instant.ofEpochSecond(
epochMicro / 1_000_000,
nanos
);

return new Timestamp(LocalDateTime.ofInstant(instant, ZoneOffset.UTC));
}

public void setNanos(int nanos) {
localDateTime = localDateTime.withNano(nanos);
}
Expand Down
16 changes: 8 additions & 8 deletions data/files/avro_timestamp.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
2012-02-21 07:08:09.123|foo:1980-12-16 07:08:09.123,bar:1998-05-07 07:08:09.123|2011-09-04 07:08:09.123,2011-09-05 07:08:09.123
2014-02-11 07:08:09.123|baz:1981-12-16 07:08:09.123|2011-09-05 07:08:09.123
1947-02-11 07:08:09.123|baz:1921-12-16 07:08:09.123|2011-09-05 07:08:09.123
8200-02-11 07:08:09.123|baz:6981-12-16 07:08:09.123|1039-09-05 07:08:09.123
1412-02-21 07:08:09.123|foo:0980-12-16 07:08:09.123,bar:0998-05-07 07:08:09.123|0011-09-04 07:08:09.123,0011-09-05 07:08:09.123
1214-02-11 07:08:09.123|baz:0981-12-16 07:08:09.123|0011-09-05 07:08:09.123
0847-02-11 07:08:09.123|baz:0921-12-16 07:08:09.123|0011-09-05 07:08:09.123
0600-02-11 07:08:09.123|baz:0981-12-16 07:08:09.123|0039-09-05 07:08:09.123
2012-02-21 07:08:09.123|foo:1980-12-16 07:08:09.123456,bar:1998-05-07 07:08:09.123456|2011-09-04 07:08:09.123456,2011-09-05 07:08:09.123456
2014-02-11 07:08:09.123456|baz:1981-12-16 07:08:09.123456|2011-09-05 07:08:09.123456
1947-02-11 07:08:09.123|baz:1921-12-16 07:08:09.123|2011-09-05 07:08:09.123456
8200-02-11 07:08:09.123456|baz:6981-12-16 07:08:09.123456|1039-09-05 07:08:09.123456
1412-02-21 07:08:09.123456|foo:0980-12-16 07:08:09.123456,bar:0998-05-07 07:08:09.123|0011-09-04 07:08:09.123456,0011-09-05 07:08:09.123
1214-02-11 07:08:09.123|baz:0981-12-16 07:08:09.123456|0011-09-05 07:08:09.123456
0847-02-11 07:08:09.123456|baz:0921-12-16 07:08:09.123456|0011-09-05 07:08:09.123456
0600-02-11 07:08:09.123|baz:0981-12-16 07:08:09.123456|0039-09-05 07:08:09.123
10 changes: 5 additions & 5 deletions ql/src/test/queries/clientpositive/avro_hybrid_mixed_timestamp.q
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ stored as avro;

INSERT INTO hybrid_table VALUES
('2012-02-21 07:08:09.123'),
('2014-02-11 07:08:09.123'),
('2014-02-11 07:08:09.123456'),
('1947-02-11 07:08:09.123'),
('8200-02-11 07:08:09.123'),
('1012-02-21 07:15:11.123'),
('1014-02-11 07:15:11.123'),
('8200-02-11 07:08:09.123456'),
('1012-02-21 07:15:11.12345'),
('1014-02-11 07:15:11.1234'),
('0947-02-11 07:15:11.123'),
('0200-02-11 07:15:11.123');
('0200-02-11 07:15:11.1234');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always refrain from modifying existing tests. Considering adding new tests.


select * from hybrid_table;

Expand Down
10 changes: 5 additions & 5 deletions ql/src/test/queries/clientpositive/avro_proleptic_mixed_timestamp.q
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@ stored as avro;

INSERT INTO hybrid_table VALUES
('2012-02-21 07:08:09.123'),
('2014-02-11 07:08:09.123'),
('1947-02-11 07:08:09.123'),
('2014-02-11 07:08:09.123456'),
('1947-02-11 07:08:09.1234'),
('8200-02-11 07:08:09.123'),
('1012-02-21 07:15:11.123'),
('1012-02-21 07:15:11.12345'),
('1014-02-11 07:15:11.123'),
('0947-02-11 07:15:11.123'),
('0200-02-11 07:15:11.123');
('0947-02-11 07:15:11.12345'),
('0200-02-11 07:15:11.123456');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refrain from changing existing tests. Add a new test for timestamp-micros explicitly mentioning its column type.


select * from hybrid_table;

Expand Down
9 changes: 9 additions & 0 deletions ql/src/test/queries/clientpositive/avro_timestamp_micros.q
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
CREATE EXTERNAL TABLE micros_table(`dt` timestamp)
STORED AS AVRO;

INSERT INTO micros_table VALUES
(cast('2024-08-09 14:08:26.326107' as timestamp)),
('2012-02-21 07:08:09.123'),
('1014-02-11 07:15:11.12345');

SELECT * FROM micros_table;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a test with timestamp-micros as a column and timestamp-millis as a column.

40 changes: 20 additions & 20 deletions ql/src/test/results/clientpositive/llap/avro_hybrid_mixed_timestamp.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,25 +14,25 @@ POSTHOOK: Output: database:default
POSTHOOK: Output: default@hybrid_table
PREHOOK: query: INSERT INTO hybrid_table VALUES
('2012-02-21 07:08:09.123'),
('2014-02-11 07:08:09.123'),
('2014-02-11 07:08:09.123456'),
('1947-02-11 07:08:09.123'),
('8200-02-11 07:08:09.123'),
('1012-02-21 07:15:11.123'),
('1014-02-11 07:15:11.123'),
('8200-02-11 07:08:09.123456'),
('1012-02-21 07:15:11.12345'),
('1014-02-11 07:15:11.1234'),
('0947-02-11 07:15:11.123'),
('0200-02-11 07:15:11.123')
('0200-02-11 07:15:11.1234')
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@hybrid_table
POSTHOOK: query: INSERT INTO hybrid_table VALUES
('2012-02-21 07:08:09.123'),
('2014-02-11 07:08:09.123'),
('2014-02-11 07:08:09.123456'),
('1947-02-11 07:08:09.123'),
('8200-02-11 07:08:09.123'),
('1012-02-21 07:15:11.123'),
('1014-02-11 07:15:11.123'),
('8200-02-11 07:08:09.123456'),
('1012-02-21 07:15:11.12345'),
('1014-02-11 07:15:11.1234'),
('0947-02-11 07:15:11.123'),
('0200-02-11 07:15:11.123')
('0200-02-11 07:15:11.1234')
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@hybrid_table
Expand All @@ -46,13 +46,13 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@hybrid_table
#### A masked pattern was here ####
2012-02-21 07:08:09.123
2014-02-11 07:08:09.123
2014-02-11 07:08:09.123456
1947-02-11 07:08:09.123
8200-02-11 07:08:09.123
1012-02-21 07:15:11.123
1014-02-11 07:15:11.123
8200-02-11 07:08:09.123456
1012-02-21 07:15:11.12345
1014-02-11 07:15:11.1234
0947-02-11 07:15:11.123
0200-02-11 07:15:11.123
0200-02-11 07:15:11.1234
PREHOOK: query: select * from hybrid_table
PREHOOK: type: QUERY
PREHOOK: Input: default@hybrid_table
Expand All @@ -62,13 +62,13 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@hybrid_table
#### A masked pattern was here ####
2012-02-21 07:08:09.123
2014-02-11 07:08:09.123
2014-02-11 07:08:09.123456
1947-02-11 07:08:09.123
8200-02-11 07:08:09.123
1012-02-21 07:15:11.123
1014-02-11 07:15:11.123
8200-02-11 07:08:09.123456
1012-02-21 07:15:11.12345
1014-02-11 07:15:11.1234
0947-02-11 07:15:11.123
0200-02-11 07:15:11.123
0200-02-11 07:15:11.1234
PREHOOK: query: drop table hybrid_table
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@hybrid_table
Expand Down
40 changes: 20 additions & 20 deletions ql/src/test/results/clientpositive/llap/avro_proleptic_mixed_timestamp.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,25 +14,25 @@ POSTHOOK: Output: database:default
POSTHOOK: Output: default@hybrid_table
PREHOOK: query: INSERT INTO hybrid_table VALUES
('2012-02-21 07:08:09.123'),
('2014-02-11 07:08:09.123'),
('1947-02-11 07:08:09.123'),
('2014-02-11 07:08:09.123456'),
('1947-02-11 07:08:09.1234'),
('8200-02-11 07:08:09.123'),
('1012-02-21 07:15:11.123'),
('1012-02-21 07:15:11.12345'),
('1014-02-11 07:15:11.123'),
('0947-02-11 07:15:11.123'),
('0200-02-11 07:15:11.123')
('0947-02-11 07:15:11.12345'),
('0200-02-11 07:15:11.123456')
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@hybrid_table
POSTHOOK: query: INSERT INTO hybrid_table VALUES
('2012-02-21 07:08:09.123'),
('2014-02-11 07:08:09.123'),
('1947-02-11 07:08:09.123'),
('2014-02-11 07:08:09.123456'),
('1947-02-11 07:08:09.1234'),
('8200-02-11 07:08:09.123'),
('1012-02-21 07:15:11.123'),
('1012-02-21 07:15:11.12345'),
('1014-02-11 07:15:11.123'),
('0947-02-11 07:15:11.123'),
('0200-02-11 07:15:11.123')
('0947-02-11 07:15:11.12345'),
('0200-02-11 07:15:11.123456')
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@hybrid_table
Expand All @@ -46,13 +46,13 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@hybrid_table
#### A masked pattern was here ####
2012-02-21 07:08:09.123
2014-02-11 07:08:09.123
1947-02-11 07:08:09.123
2014-02-11 07:08:09.123456
1947-02-11 07:08:09.1234
8200-02-11 07:08:09.123
1012-02-21 07:15:11.123
1012-02-21 07:15:11.12345
1014-02-11 07:15:11.123
0947-02-11 07:15:11.123
0200-02-11 07:15:11.123
0947-02-11 07:15:11.12345
0200-02-11 07:15:11.123456
PREHOOK: query: select * from hybrid_table
PREHOOK: type: QUERY
PREHOOK: Input: default@hybrid_table
Expand All @@ -62,13 +62,13 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@hybrid_table
#### A masked pattern was here ####
2012-02-21 07:08:09.123
2014-02-11 07:08:09.123
1947-02-11 07:08:09.123
2014-02-11 07:08:09.123456
1947-02-11 07:08:09.1234
8200-02-11 07:08:09.123
1012-02-21 07:15:11.123
1012-02-21 07:15:11.12345
1014-02-11 07:15:11.123
0947-02-11 07:15:11.123
0200-02-11 07:15:11.123
0947-02-11 07:15:11.12345
0200-02-11 07:15:11.123456
PREHOOK: query: drop table hybrid_table
PREHOOK: type: DROPTABLE
PREHOOK: Input: default@hybrid_table
Expand Down
54 changes: 27 additions & 27 deletions ql/src/test/results/clientpositive/llap/avro_timestamp.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,14 +75,14 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@avro_timestamp
POSTHOOK: Input: default@avro_timestamp@p1=2/p2=2014-09-26 07%3A08%3A09.123
#### A masked pattern was here ####
2012-02-21 07:08:09.123 {"bar":"1998-05-07 07:08:09.123","foo":"1980-12-16 07:08:09.123"} ["2011-09-04 07:08:09.123","2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
2014-02-11 07:08:09.123 {"baz":"1981-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1947-02-11 07:08:09.123 {"baz":"1921-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
8200-02-11 07:08:09.123 {"baz":"6981-12-16 07:08:09.123"} ["1039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1412-02-21 07:08:09.123 {"bar":"0998-05-07 07:08:09.123","foo":"0980-12-16 07:08:09.123"} ["0011-09-04 07:08:09.123","0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1214-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123"} ["0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
0847-02-11 07:08:09.123 {"baz":"0921-12-16 07:08:09.123"} ["0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
0600-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123"} ["0039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
2012-02-21 07:08:09.123 {"bar":"1998-05-07 07:08:09.123456","foo":"1980-12-16 07:08:09.123456"} ["2011-09-04 07:08:09.123456","2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
2014-02-11 07:08:09.123456 {"baz":"1981-12-16 07:08:09.123456"} ["2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
1947-02-11 07:08:09.123 {"baz":"1921-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
8200-02-11 07:08:09.123456 {"baz":"6981-12-16 07:08:09.123456"} ["1039-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
1412-02-21 07:08:09.123456 {"bar":"0998-05-07 07:08:09.123","foo":"0980-12-16 07:08:09.123456"} ["0011-09-04 07:08:09.123456","0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1214-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123456"} ["0011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
0847-02-11 07:08:09.123456 {"baz":"0921-12-16 07:08:09.123456"} ["0011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
0600-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123456"} ["0039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
PREHOOK: query: SELECT d, COUNT(d) FROM avro_timestamp GROUP BY d
PREHOOK: type: QUERY
PREHOOK: Input: default@avro_timestamp
Expand All @@ -96,11 +96,11 @@ POSTHOOK: Input: default@avro_timestamp@p1=2/p2=2014-09-26 07%3A08%3A09.123
0600-02-11 07:08:09.123 1
1214-02-11 07:08:09.123 1
1947-02-11 07:08:09.123 1
2014-02-11 07:08:09.123 1
0847-02-11 07:08:09.123 1
1412-02-21 07:08:09.123 1
0847-02-11 07:08:09.123456 1
1412-02-21 07:08:09.123456 1
2012-02-21 07:08:09.123 1
8200-02-11 07:08:09.123 1
2014-02-11 07:08:09.123456 1
8200-02-11 07:08:09.123456 1
PREHOOK: query: SELECT * FROM avro_timestamp WHERE d!='1947-02-11 07:08:09.123'
PREHOOK: type: QUERY
PREHOOK: Input: default@avro_timestamp
Expand All @@ -111,13 +111,13 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@avro_timestamp
POSTHOOK: Input: default@avro_timestamp@p1=2/p2=2014-09-26 07%3A08%3A09.123
#### A masked pattern was here ####
2012-02-21 07:08:09.123 {"bar":"1998-05-07 07:08:09.123","foo":"1980-12-16 07:08:09.123"} ["2011-09-04 07:08:09.123","2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
2014-02-11 07:08:09.123 {"baz":"1981-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
8200-02-11 07:08:09.123 {"baz":"6981-12-16 07:08:09.123"} ["1039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1412-02-21 07:08:09.123 {"bar":"0998-05-07 07:08:09.123","foo":"0980-12-16 07:08:09.123"} ["0011-09-04 07:08:09.123","0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1214-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123"} ["0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
0847-02-11 07:08:09.123 {"baz":"0921-12-16 07:08:09.123"} ["0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
0600-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123"} ["0039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
2012-02-21 07:08:09.123 {"bar":"1998-05-07 07:08:09.123456","foo":"1980-12-16 07:08:09.123456"} ["2011-09-04 07:08:09.123456","2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
2014-02-11 07:08:09.123456 {"baz":"1981-12-16 07:08:09.123456"} ["2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
8200-02-11 07:08:09.123456 {"baz":"6981-12-16 07:08:09.123456"} ["1039-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
1412-02-21 07:08:09.123456 {"bar":"0998-05-07 07:08:09.123","foo":"0980-12-16 07:08:09.123456"} ["0011-09-04 07:08:09.123456","0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1214-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123456"} ["0011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
0847-02-11 07:08:09.123456 {"baz":"0921-12-16 07:08:09.123456"} ["0011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
0600-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123456"} ["0039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
PREHOOK: query: SELECT * FROM avro_timestamp WHERE d<'2014-12-21 07:08:09.123'
PREHOOK: type: QUERY
PREHOOK: Input: default@avro_timestamp
Expand All @@ -128,13 +128,13 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@avro_timestamp
POSTHOOK: Input: default@avro_timestamp@p1=2/p2=2014-09-26 07%3A08%3A09.123
#### A masked pattern was here ####
2012-02-21 07:08:09.123 {"bar":"1998-05-07 07:08:09.123","foo":"1980-12-16 07:08:09.123"} ["2011-09-04 07:08:09.123","2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
2014-02-11 07:08:09.123 {"baz":"1981-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1947-02-11 07:08:09.123 {"baz":"1921-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1412-02-21 07:08:09.123 {"bar":"0998-05-07 07:08:09.123","foo":"0980-12-16 07:08:09.123"} ["0011-09-04 07:08:09.123","0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1214-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123"} ["0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
0847-02-11 07:08:09.123 {"baz":"0921-12-16 07:08:09.123"} ["0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
0600-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123"} ["0039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
2012-02-21 07:08:09.123 {"bar":"1998-05-07 07:08:09.123456","foo":"1980-12-16 07:08:09.123456"} ["2011-09-04 07:08:09.123456","2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
2014-02-11 07:08:09.123456 {"baz":"1981-12-16 07:08:09.123456"} ["2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
1947-02-11 07:08:09.123 {"baz":"1921-12-16 07:08:09.123"} ["2011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
1412-02-21 07:08:09.123456 {"bar":"0998-05-07 07:08:09.123","foo":"0980-12-16 07:08:09.123456"} ["0011-09-04 07:08:09.123456","0011-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
1214-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123456"} ["0011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
0847-02-11 07:08:09.123456 {"baz":"0921-12-16 07:08:09.123456"} ["0011-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
0600-02-11 07:08:09.123 {"baz":"0981-12-16 07:08:09.123456"} ["0039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
PREHOOK: query: SELECT * FROM avro_timestamp WHERE d>'8000-12-01 07:08:09.123'
PREHOOK: type: QUERY
PREHOOK: Input: default@avro_timestamp
Expand All @@ -145,4 +145,4 @@ POSTHOOK: type: QUERY
POSTHOOK: Input: default@avro_timestamp
POSTHOOK: Input: default@avro_timestamp@p1=2/p2=2014-09-26 07%3A08%3A09.123
#### A masked pattern was here ####
8200-02-11 07:08:09.123 {"baz":"6981-12-16 07:08:09.123"} ["1039-09-05 07:08:09.123"] 2 2014-09-26 07:08:09.123
8200-02-11 07:08:09.123456 {"baz":"6981-12-16 07:08:09.123456"} ["1039-09-05 07:08:09.123456"] 2 2014-09-26 07:08:09.123
36 changes: 36 additions & 0 deletions ql/src/test/results/clientpositive/llap/avro_timestamp_micros.q.out
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
PREHOOK: query: CREATE EXTERNAL TABLE micros_table(`dt` timestamp)
STORED AS AVRO
PREHOOK: type: CREATETABLE
PREHOOK: Output: database:default
PREHOOK: Output: default@micros_table
POSTHOOK: query: CREATE EXTERNAL TABLE micros_table(`dt` timestamp)
STORED AS AVRO
POSTHOOK: type: CREATETABLE
POSTHOOK: Output: database:default
POSTHOOK: Output: default@micros_table
PREHOOK: query: INSERT INTO micros_table VALUES
(cast('2024-08-09 14:08:26.326107' as timestamp)),
('2012-02-21 07:08:09.123'),
('1014-02-11 07:15:11.12345')
PREHOOK: type: QUERY
PREHOOK: Input: _dummy_database@_dummy_table
PREHOOK: Output: default@micros_table
POSTHOOK: query: INSERT INTO micros_table VALUES
(cast('2024-08-09 14:08:26.326107' as timestamp)),
('2012-02-21 07:08:09.123'),
('1014-02-11 07:15:11.12345')
POSTHOOK: type: QUERY
POSTHOOK: Input: _dummy_database@_dummy_table
POSTHOOK: Output: default@micros_table
POSTHOOK: Lineage: micros_table.dt SCRIPT []
PREHOOK: query: SELECT * FROM micros_table
PREHOOK: type: QUERY
PREHOOK: Input: default@micros_table
#### A masked pattern was here ####
POSTHOOK: query: SELECT * FROM micros_table
POSTHOOK: type: QUERY
POSTHOOK: Input: default@micros_table
#### A masked pattern was here ####
2024-08-09 14:08:26.326107
2012-02-21 07:08:09.123
1014-02-11 07:15:11.12345
Loading
Loading