Scala and AWS managed ElasticSearch

AWS offer a managed ElasticSearch service. It exposes an HTTP endpoint for interacting with ElasticSearch and requires authentication via AWS Identity Access Management.

Elastic4s offers a neat DSL and Scala client for ElasticSearch. This post details how to use it with AWS’s managed ElasticSearch service.

Creating a request signer

Using the aws-signing-request-interceptor library its easy to create an HttpRequestInterceptor which can be later added to the HttpClient used by Elastic4s for making the calls to ElasticSearch

private def createAwsSigner(config: Config): AWSSigner = {
  import com.gilt.gfc.guava.GuavaConversions._

  val awsCredentialsProvider = new DefaultAWSCredentialsProviderChain
  val service = config.getString("service")
  val region = config.getString("region")
  val clock: Supplier[LocalDateTime] = () =>"UTC"))
  new AWSSigner(awsCredentialsProvider, region, service, clock)

Creating an HTTP Client and intercepting the requests

The ElasticSearch RestClientBuilder allows for registering a callback to modify the customise the HttpAsyncClientBuilder enabling registering the interceptor to sign the requests.

The callback can be created by implementing the HttpClientConfigCallback interface as follows:

private val esConfig = config.getConfig("elasticsearch")

private class AWSSignerInteceptor extends HttpClientConfigCallback {
  override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
    httpClientBuilder.addInterceptorLast(new AWSSigningRequestInterceptor(createAwsSigner(esConfig)))

Finally, an Elastic4s client can be created with the interceptor registered:

private def createEsHttpClient(config: Config): HttpClient = {
  val hosts = ElasticsearchClientUri(config.getString("uri")) {
    case (host, port) =>
      new HttpHost(host, port, "http")
  }"Creating HTTP client on ${hosts.mkString(",")}")

  val client = RestClient.builder(hosts: _*)
    .setHttpClientConfigCallback(new AWSSignerInteceptor)

Full Example on GitHub