A log record whose body or attributes carry bytes that are not valid UTF-8 aborts the process when the Elasticsearch exporter serialises it. The recordable stores the value as it was given, nothing on the way in checks it, and Export() is noexcept, so the exception dump() raises has nowhere to go.
Describe your environment
main at 60c3d11, Linux 6.12 x86_64, gcc 14.2.0, nlohmann/json 3.11.3, -DWITH_ELASTICSEARCH=ON, plain CMake build with the defaults. No sanitizer needed.
Steps to reproduce
Drop this into exporters/elasticsearch/test/es_log_record_exporter_test.cc, next to the fakes already there. The client never has to answer, the process is gone before the request is sent.
TEST(ElasticsearchLogsExporterTests, ExportingARecordWithInvalidUtf8)
{
auto client = std::make_shared<FakeHttpClient>([](http_client::EventHandler &handler) {
FakeResponse response(200, kCompactSuccess);
handler.OnResponse(response);
});
logs_exporter::ElasticsearchExporterOptions options;
logs_exporter::ElasticsearchLogRecordExporter exporter(options, client);
auto record = exporter.MakeRecordable();
std::string body = "payload ";
body += "\xC3\x28"; // a two byte sequence that is not valid UTF-8
record->SetBody(body);
exporter.Export(nostd::span<std::unique_ptr<sdklogs::Recordable>>(&record, 1));
}What is the expected behavior?
The export fails, or the exporter replaces what it cannot encode, and the application keeps running. A log record is application data. Which bytes end up in one is not something the application always controls: a message built from a file path, a truncated multibyte character, a string read in another encoding, or any payload passed through as text will do it.
What is the actual behavior?
[ RUN ] ElasticsearchLogsExporterTests.ExportingARecordWithInvalidUtf8
terminate called after throwing an instance of 'nlohmann::json_abi_v3_11_3::detail::type_error'
what(): [json.exception.type_error.316] invalid UTF-8 byte at index 9: 0x28
Aborted (core dumped)
Exit status 134. The whole process, not the export.
Where it comes apart
ElasticSearchRecordable::WriteValue stores the value as given (exporters/elasticsearch/src/es_log_recordable.cc:87 and :93):
Nothing in the exporter validates or transcodes it. grep -rn 'utf8\|error_handler' exporters/elasticsearch/ finds nothing.
ElasticsearchLogRecordExporter::Export is declared noexcept (exporters/elasticsearch/src/es_log_record_exporter.cc:394) and builds the bulk body with (:432):
body += json_record->GetJSON().dump() + "\n";
dump() raises type_error.316 when a string in the document is not valid UTF-8. Leaving a noexcept function through an exception calls std::terminate, so the throw becomes an abort.
This is the serialising direction only. The response parsing side is not affected: nlohmann::json::parse with allow_exceptions false rejects a body carrying invalid UTF-8 and reports it as discarded, so those bytes never reach a dump() there. I checked that separately rather than assuming it.
Two shapes of fix
nlohmann can be told not to raise. dump(-1, ' ', false, nlohmann::json::error_handler_t::replace) substitutes U+FFFD for each byte it cannot encode, which keeps the record and the export, and loses only the bytes that were never valid to begin with. That is a one line change at the call site and it also covers attributes, the resource, and the scope name, since they all go through the same document.
Validating on the way in instead, in WriteValue, would let the exporter say which field was rejected, but it needs a UTF-8 check the SDK does not currently have and it has to decide what to do with a record it will not send.
I have no preference between them and am happy to send a PR for whichever the maintainers would rather have.
Additional context
I found this while auditing the same file for #4297, which changes how the bulk response is read. It is unrelated to that PR and reproduces on main on its own.
Other exporters build their payload the same way, so the same shape may exist elsewhere. I have only measured the Elasticsearch one and have not gone looking, so this issue claims nothing about the others.
A log record whose body or attributes carry bytes that are not valid UTF-8 aborts the process when the Elasticsearch exporter serialises it. The recordable stores the value as it was given, nothing on the way in checks it, and
Export()isnoexcept, so the exceptiondump()raises has nowhere to go.Describe your environment
mainat 60c3d11, Linux 6.12 x86_64, gcc 14.2.0, nlohmann/json 3.11.3,-DWITH_ELASTICSEARCH=ON, plain CMake build with the defaults. No sanitizer needed.Steps to reproduce
Drop this into
exporters/elasticsearch/test/es_log_record_exporter_test.cc, next to the fakes already there. The client never has to answer, the process is gone before the request is sent.What is the expected behavior?
The export fails, or the exporter replaces what it cannot encode, and the application keeps running. A log record is application data. Which bytes end up in one is not something the application always controls: a message built from a file path, a truncated multibyte character, a string read in another encoding, or any payload passed through as text will do it.
What is the actual behavior?
Exit status 134. The whole process, not the export.
Where it comes apart
ElasticSearchRecordable::WriteValuestores the value as given (exporters/elasticsearch/src/es_log_recordable.cc:87and:93):Nothing in the exporter validates or transcodes it.
grep -rn 'utf8\|error_handler' exporters/elasticsearch/finds nothing.ElasticsearchLogRecordExporter::Exportis declarednoexcept(exporters/elasticsearch/src/es_log_record_exporter.cc:394) and builds the bulk body with (:432):body += json_record->GetJSON().dump() + "\n";dump()raisestype_error.316when a string in the document is not valid UTF-8. Leaving anoexceptfunction through an exception callsstd::terminate, so the throw becomes an abort.This is the serialising direction only. The response parsing side is not affected:
nlohmann::json::parsewithallow_exceptionsfalse rejects a body carrying invalid UTF-8 and reports it as discarded, so those bytes never reach adump()there. I checked that separately rather than assuming it.Two shapes of fix
nlohmann can be told not to raise.
dump(-1, ' ', false, nlohmann::json::error_handler_t::replace)substitutes U+FFFD for each byte it cannot encode, which keeps the record and the export, and loses only the bytes that were never valid to begin with. That is a one line change at the call site and it also covers attributes, the resource, and the scope name, since they all go through the same document.Validating on the way in instead, in
WriteValue, would let the exporter say which field was rejected, but it needs a UTF-8 check the SDK does not currently have and it has to decide what to do with a record it will not send.I have no preference between them and am happy to send a PR for whichever the maintainers would rather have.
Additional context
I found this while auditing the same file for #4297, which changes how the bulk response is read. It is unrelated to that PR and reproduces on
mainon its own.Other exporters build their payload the same way, so the same shape may exist elsewhere. I have only measured the Elasticsearch one and have not gone looking, so this issue claims nothing about the others.