4520 – Add metrics to code and create Honeycomb status dashboard: parser requests [WIP] (#458)

We are now sending metrics related to parsing requests. We are keeping track of: service_name, parser_name, parsed_host and parsing_status.

What we want to answer:
    total of parsing requests per x amount of time
    total of parsing requests per x amount of time grouped by partner [TBD: not possible to do right now]
    total of parsing requests per x amount of time grouped by parser
    total of parsing requests of an unique host, which does not have a dedicated parser, per x amount of time
    total of errors and total of successes of our parsing requests per x amount of time

Why we are sending the data we are sending:

I ended up splitting the metrics so we can get more granular data and more generic data. I did this because it was proving difficult to get the information we wanted on the Honeycomb side.

For some cases we wanted a SUM of the value send by the metric, but I think since the otel-collector sends the scraped data every 15 seconds we end up with a bunch of events. The SUM doesn't really show what we need. We can zoom in into specific points in time, and I have set the default granularity to match the otel-collector. All that helps, but doesn't get 100% what we would like.

I ended up using MAX instead of SUM. But that was an issue when we had more granular metrics and wanted to know, for example, the amount of parsing requests to the page parser, without breaking it up by host.

If we made 2 parsing requests to page but to different hosts, those would be split. And we would get the MAX 1 instead of 2.


References: 4396, 4520
PR: 458

Author: vasconsaurus

app/models/media.rb

@@ -86,7 +86,7 @@ def as_json(options = {})
     end
     archive_if_conditions_are_met(options, id, cache)
     Metrics.schedule_fetching_metrics_from_facebook(self.data, self.url, ApiKey.current&.id)
-    MetricsService.increment_counter(:media_request_total, labels: { service: 'pender', parser: self.data['provider'] })
+    parser_requests_metrics
     cache.read(id, :json) || cleanup_data_encoding(data)
   end
 
@@ -326,4 +326,20 @@ def archive_if_conditions_are_met(options, id, cache)
         self.archive(options.delete(:archivers))
     end
   end
+
+  def parser_requests_metrics
+    url = RequestHelper.normalize_url(self.url)
+
+    MetricsService.increment_counter(:pender_parser_requests_total)
+    MetricsService.increment_counter(:pender_parser_requests_per_parser, labels: { parser_name: data[:provider], parsing_status: data[:error].nil? ?  'success' : 'error' })
+    MetricsService.increment_counter(:pender_parser_requests, labels: { parser_name: data[:provider], parsed_host: URI(url).host, parsing_status: data[:error].nil? ?  'success' : 'error' })
+    if data[:error].nil?
+      MetricsService.increment_counter(:pender_parser_requests_success)
+      MetricsService.increment_counter(:pender_parser_requests_success_per_parser, labels: { parser_name: data[:provider] })
+    else
+      MetricsService.increment_counter(:pender_parser_requests_error)
+      MetricsService.increment_counter(:pender_parser_requests_error_per_parser, labels: { parser_name: data[:provider] })
+    end
+  end
+
 end

config/initializers/metrics_registry.rb

@@ -1,3 +1,11 @@
 Rails.application.reloader.to_prepare do
-  MetricsService.custom_counter(:media_request_total, 'Count every request made', labels: [:service, :parser])
+  MetricsService.custom_counter(:pender_parser_requests_total, 'Count parsing requests, gets the full total - does not break them by labels', labels: [:service_name])
+  MetricsService.custom_counter(:pender_parser_requests_success, 'Count successful parsing requests', labels: [:service_name])
+  MetricsService.custom_counter(:pender_parser_requests_error, 'Count errored parsing requests', labels: [:service_name])
+
+  MetricsService.custom_counter(:pender_parser_requests_per_parser, 'Count parsing requests per parser', labels: [:service_name, :parser_name, :parsing_status])
+  MetricsService.custom_counter(:pender_parser_requests_success_per_parser, 'Count successful parsing requests per parser', labels: [:service_name, :parser_name])
+  MetricsService.custom_counter(:pender_parser_requests_error_per_parser, 'Count errored parsing requests per parser', labels: [:service_name, :parser_name])
+
+  MetricsService.custom_counter(:pender_parser_requests, 'Count parsing requests - broken by labels', labels: [:service_name, :parser_name, :parsed_host, :parsing_status])
 end

lib/metrics_service.rb

@@ -6,7 +6,8 @@ def custom_counter(name, description, labels: [])
       counter = Prometheus::Client::Counter.new(
         name, 
         docstring: description,
-        labels: labels
+        labels: labels,
+        preset_labels: { service_name: 'pender' }
         )
       prometheus_registry.register(counter)
     end

test/integration/parsers/page_item_test.rb

@@ -26,24 +26,6 @@ class PageItemIntegrationTest < ActiveSupport::TestCase
     assert_equal '', data['username']
   end
 
-  test "should parse url with arabic or already encoded chars" do
-    urls = [
-      'https://www.aljazeera.net/news/2023/2/9/الشرطة-السويدية-ترفض-منح-إذن-لحرق',
-      'https://www.aljazeera.net/news/2023/2/9/%D8%A7%D9%84%D8%B4%D8%B1%D8%B7%D8%A9-%D8%A7%D9%84%D8%B3%D9%88%D9%8A%D8%AF%D9%8A%D8%A9-%D8%AA%D8%B1%D9%81%D8%B6-%D9%85%D9%86%D8%AD-%D8%A5%D8%B0%D9%86-%D9%84%D8%AD%D8%B1%D9%82'
-    ]
-    urls.each do |url|
-      m = create_media url: url
-      data = m.as_json
-      assert_equal 'الشرطة السويدية ترفض منح إذن جديد لحرق المصحف الشريف أمام السفارة التركية.. فما السبب؟', data['title']
-      assert_equal 'رفضت الشرطة السويدية منح إذن لحرق المصحف الشريف أمام السفارة التركية، قائلة إن ذلك من شأنه “إثارة اضطرابات خطيرة للأمن القومي”.', data['description']
-      assert_equal '', data['published_at']
-      assert_equal '', data['username']
-      assert_match /^https?:\/\/www\.aljazeera\.net$/, data['author_url']
-      assert_nil data['error']
-      assert_not_nil data['picture']
-    end
-  end
-
   test "should store metatags in an Array" do
     m = create_media url: 'https://www.nytimes.com/2017/06/14/us/politics/mueller-trump-special-counsel-investigation.html'
     data = m.as_json
@@ -105,18 +87,6 @@ class PageItemIntegrationTest < ActiveSupport::TestCase
     assert !data['picture'].blank?
   end
 
-  test "should parse urls without utf encoding" do
-    urls = [
-      'https://www.aljazeera.net/news/2024/1/24/شهيد-بالضفة-والاحتلال-يعتقل-طفلا-حرر',
-      'https://www.aljazeera.net/news/2024/1/24/%D8%B4%D9%87%D9%8A%D8%AF-%D8%A8%D8%A7%D9%84%D8%B6%D9%81%D8%A9-%D9%88%D8%A7%D9%84%D8%A7%D8%AD%D8%AA%D9%84%D8%A7%D9%84-%D9%8A%D8%B9%D8%AA%D9%82%D9%84-%D8%B7%D9%81%D9%84%D8%A7-%D8%AD%D8%B1%D8%B1',
-    ]
-    urls.each do |url|
-      m = create_media url: url
-      data = m.as_json
-      assert data['error'].nil?
-    end
-  end
-
   test "should use original url when redirected page requires cookie" do
     RequestHelper.stubs(:get_html).returns(Nokogiri::HTML("<meta property='og:url' content='https://www.tandfonline.com/action/cookieAbsent'><meta name='pbContext' content=';wgroup:string:Publication Websites;website:website:TFOPB;page:string:Cookie Absent'>"))
     url = 'https://doi.org/10.1080/10584609.2019.1619639'

test/lib/metrics_service_test.rb

@@ -3,20 +3,20 @@
 class MetricsServiceTest < ActiveSupport::TestCase
   test "custom_counter works for real" do
     assert_nothing_raised do
-      MetricsService.custom_counter(:custom_counter, 'Custom counter test', labels: [:test])
+      MetricsService.custom_counter(:custom_counter, 'Custom counter test', labels: [:service_name, :test])
     end
   end
 
   test "increment_counter works for real" do
-    MetricsService.custom_counter(:custom_counter_2, 'Custom counter test', labels: [:test])
+    MetricsService.custom_counter(:custom_counter_2, 'Custom counter test', labels: [:service_name, :test])
 
     assert_nothing_raised do
       MetricsService.increment_counter(:custom_counter_2, labels: [:test])
     end
   end
 
   test "get_counter works for real" do
-    custom_counter = MetricsService.custom_counter(:custom_counter_3, 'Custom counter test', labels: [:test])
+    custom_counter = MetricsService.custom_counter(:custom_counter_3, 'Custom counter test', labels: [:service_name, :test])
 
     assert_nothing_raised do
       MetricsService.get_counter(custom_counter, labels: [:test])