Turn down of v2 log lists by google on 7th June 2023

[dantuswaroop]

Hi team,

Google has announced that they will turn down the v2 version of log listing by 7th June 2023, we want to upgrade the apps before that.
Here is the conversation link.
https://groups.google.com/g/certificate-transparency/c/38Lr9K46cCA

Just wanted to know the following points.

1. When will the new version of the library be available, pointing to v3 version
2. Google also warned that the log_list version will change at anytime without notice and that will break the app. How can we recover or counter that with the new version of the app.
3. If log_listing version is again updated by google, will that need another library upgrade and app upgrade ?

Thanks
Swaroop

[mattmook]

1. Should be within the next 1 or 2 weeks; there's one PR ready to merge, one more bug to investigate in a little more detail and some automation to setup.

2. Caching has been re-implemented and made a lot more robust. For example the library now embeds a hard coded log list so it always has a fallback. The implementation is closer to how Chrome works, if the library cannot update the log list for 70 days then the best it can do is disable the CT checks. The plan is for more regular updates of the library to keep the embedded log list in sync so as long as apps keep the library up to date that 70 days will keep on rolling - a disk cache being used also extends that 70 days too.
   The new implementation additionally reduces the risk of replay attacks.

3. Hard to know where they plan to take the file; potentially but it depends on the changes to the format and what that means for the app. v3 wasn't that different and the library did work with the file without any changes other than to the URL.

One solution to 2 and 3 is to host the log list file ourselves to give an additional level of separation; Google isn't the only source for log-list.json so this would allow internal redirections. this would have to be a paid service though but then could come with paid support too if so desired - its not something I've particularly thought about though but as an open source library there's only so much free time I'm willing to give especially given the project has never had a single sponsor.

Of course I do wonder how short lived that might be; maybe one day Google will actually release their own tooling around the checks rather than Conscrypts broken/unusable/hacky implementation.

[barnhill]

Its my understanding that this has now been released with this pull request #47. Can this issue be closed?

[mattmook]

hi @barnhill, yes; left it open for visibility, although realise my comment is out of date now!

1. Released already
2. Comment about caching still stands. I've setup a GitHub action to download the latest log list file daily and automatically tags a patch version weekly on a Wednesday

[gilbertparreno]

Hi @mattmook, right now we can't update to new library version which is v2.4.3 and right we're still using v1.1.1, would it be okay if we stick on using the v1.1.1 and use https://www.gstatic.com/ct/log_list/v3/?

See code below:

certificateTransparencyInterceptor { setLogListService( LogListDataSourceFactory.createLogListService( "https://www.gstatic.com/ct/log_list/v3/" ) ) val certifiedHosts = mutableListOf<String>().apply { // urls should be added here } certifiedHosts.forEach { +it } }

[mattmook]

hi @gilbertparreno, no immediate concerns you just miss out on the improvements in the latest version - for one the better fail over as described above.

[maxkohne]

Important changes to the v2 CT log list

TLDR: Google is no longer sunsetting the v2 CT endpoint on June 7th due to slow Android adoption rate to v3. They will continue supporting v2 knowing that it will not be secure while highly recommending a v3 migration. They are still investigating a 1st party solution for an Android CT library.

Hi all,

Today we're announcing some important changes to our deprecation plan of the v2 log list URL.

Usage of the v2 log list URL in third-party Android applications has not decreased at the rate we had hoped, with most of these apps failing to work correctly when the log list is missing. To avoid this breakage, the v2 log list will remain available beyond 2023-06-07, but with reduced security guarantees. We continue to ask you to not use the v2 log list, and to migrate away from it to allow for a future turndown.

Going forward, while logs will be added to this list (once they reach the USABLE state in Chrome), logs will never be removed, even if they are Retired in Chrome. This change ensures that applications using the v2 URL are able to validate SCTs (albeit with fewer security guarantees) until those applications have received updates. Simultaneously, this change ensures that Chrome has the freedom necessary to retire logs without risking breakage to these applications. This list is still offered without any SLA.

Over time, the v2 and v3 log list URLs will diverge, and the v2 URL will not reflect Chrome's perspective on the current state of the CT ecosystem. Certificate issuers, monitors, and other actors that need to track Chrome's view of usable logs must still migrate to the v3 log list.

We believe that enforcing CT effectively and safely requires a coordinated maintenance of client enforcing code, a log list, and their respective update mechanism. To the best of our knowledge, there isn’t such an end-to-end CT solution for Android today. We’re committed to making CT enforcement easier on Android in the future, and hope to share more about this soon. Stay tuned on the certificate-transparency mailing list.

Cheers,
Google CT team, in collaboration with the Chrome team

— https://groups.google.com/g/certificate-transparency/c/4YWS71teTY0

[mattmook]

They are still investigating a 1st party solution for an Android CT library.

They’ve had half written code in conscrypt for 3/4 years so it couldn’t come sooner imo. iOS have had built in support for about that amount of time too!

[chintan-soni-cko]

@mattmook Really appreciate the work you have done by releasing this fantastic library. Can you please advise if weekly updating the log list and updating the library weekly means we need as a consumer to update every time when timestamp change from Loglist?

[mattmook]

@chintan-soni-cko it's complicated. Generally not, I'm doing it partly because it keeps the library up-to-date.

The file comes into play when the library is unable to download a newer version of the file for whatever reason (main scenario here is saying that Google no longer hosts the file or it no longer parses correctly) and does not have a version of the file cached locally. This will generally be on the first use of an app. The file stays valid for 90 days; I view this as saying as long as you push an update out every 90 days then you're unlikely to see any problems.