Managing web crawler traffic with reCAPTCHA v3

As web and AI crawlers have become more aggressive we needed to address this behavior in order to maintain LUNA’s availability.

The LUNA Viewer was designed to encourage end users to drill down into a collection’s content using the Who What Where When facets. Web crawlers also used these facets to discover the content of collections. This was good for Search Engines like Google to discover the content in LUNA.

We have added an option that will allow end users to continue to use the facets and to excluded non-human access. This option makes use of Google’s reCAPTCHA v3

To implement this feature, you will need a few things

LUNA version 7.5.5.6+
Register your site with Google’s reCAPTCHA.
- https://www.google.com/recaptcha/admin/create
  - You probably want to use a Google account that is associated with your organization
  - Score based (v3)
  - Add the domain name that LUNA is running on
  - You will need the Site Key and the Secret Key for LUNA.

Now you need to open the LUNA Administrator as a super admin

Select the Application Configuration tab.

Add the following three new attributes using the “Add Application Configuration Attribute”

recaptchaV3Enabled    true
recaptchaV3SecretKey    {your_secret_key}
recaptchaV3SiteKey    {your_site_key}

Using the keys that were generated in Google reCAPTCHA

Once these are added. Have your LUNA system administrator restart LUNA’s tomcat service.

The last step is to Restrict Facets in the credential(s) that are public facing. This is most likely the Anonymous Credential.

For reference:

https://lunaimaging.atlassian.net/wiki/x/sAIK