[AXIS2] Throttle upload bandwidth during sending SOAP messages

[robertlazarski]

Guten Tag robertlazarski,

The Apache httpcomponents lib from the client [...] can tune some of
the rate behavior.

So "7.1. Custom client connections" seem to be the way to go: Provide
a custom HttpConnectionFactory and make Axis2 to use that, while I've
already provided a custom connection manager to Axis2 in the past with
non-default settings. What I'm missing currently is the part where the
request body gets serialized, "DefaultHttpRequestWriterFactory" reads
like it only handles headers...
https://hc.apache.org/httpcomponents-client-ga/
tutorial/html/advanced.html#d5e913

I see. If you think your code may be generally useful please consider
submitting patches.

Since request.getInputStream() can only be called once, a trick is to read
it in a filter first so then the stacktrace shows the read location.

In my funky GSON setup I am currently working on, that is occurring in
AxisServlet.doPost() .

Regards,
Robert

Really though, I would first look at nginx or some type of proxy
server first as that is perhaps the best tool for the job.

I'll keep that in mind, already had a quick look at Squid, which my
customer is using anyway and that sounds promising as well. Using
something in my app/Axis2 might provide the benefit of less admin
overhead for my customer, though. Some parts of his infrastructure are
even outsourced and can't be influenced easily.
Mit freundlichen GrÌßen,
Thorsten Schöning
--
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - GeschÀftsfÌhrer: Andreas Muchow
---------------------------------------------------------------------

[Thorsten Schöning]

Guten Tag Thorsten Schöning,

So "7.1. Custom client connections" seem to be the way to go: Provide
a custom HttpConnectionFactory and make Axis2 to use that, while I've
already provided a custom connection manager to Axis2 in the past with
non-default settings. What I'm missing currently is the part where the
request body gets serialized, "DefaultHttpRequestWriterFactory" reads
like it only handles headers...
https://hc.apache.org/httpcomponents-client-ga/tutorial/html/advanced.html#d5e913

The important part to understand seems to be that the connection
factory provides some instances of "HttpClientConnection" in the end
and those distinguishe between serializing headers and bodies:

https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpClientConnection.html#sendRequestHeader(org.apache.http.HttpRequest)
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/http/HttpClientConnection.html#sendRequestEntity(org.apache.http.HttpEntityEnclosingRequest)

The request writers are only used in "sendRequestHeader", while bodies
are serialized using whatever body-containing object is given. In case
of Axis2 this seems to be "AxisRequestEntityImpl", which takes care of
serializing according to some "MessageFormatter", which itself exists
for various different formats like JSON and SOAP and things like MTOM
vs. inline vs. SOAP with attachments are handled somewhere in that
chain as well...

In the end, every data is written to a stream created within
"sendRequestEntity" and that seems to be a good place to put a rate

@Override
public void sendRequestEntity(final HttpEntityEnclosingRequest request)
throws HttpException, IOException {
Args.notNull(request, "HTTP request");
ensureOpen();
final HttpEntity entity = request.getEntity();
if (entity == null) {
return;
}
final OutputStream outstream = prepareOutput(request);
entity.writeTo(outstream);
outstream.close();
}
protected OutputStream createOutputStream(
final long len,
final SessionOutputBuffer outbuffer) {
if (len == ContentLengthStrategy.CHUNKED) {
return new ChunkedOutputStream(2048, outbuffer);
} else if (len == ContentLengthStrategy.IDENTITY) {
return new IdentityOutputStream(outbuffer);
} else {
return new ContentLengthOutputStream(outbuffer, len);
}
}
protected OutputStream prepareOutput(final HttpMessage message) throws HttpException {
final long len = this.outgoingContentStrategy.determineLength(message);
return createOutputStream(len, this.outbuffer);
}

So, the connection factory is needed to provide custom client
connections and one needs to create one which overrides
"createOutputStream" by calling the super implementation and
afterwards wrapping the resulting "OutputStream" once more in some
"RateLimitOutputStream" or such.

From my understanding, that "RateLimitOutputStream" would "only" need
to make sure that writes are somewhat small, so that not e.g. 1 GB of
data is transferred during one call, but smaller blocks of some kB or
MB or such. Each transfer of some small block is forwarded to the
wrapped "OutputStream" after some rate limiting resources have been
aquired like in the following example:

https://stackoverflow.com/a/6271935/2055163

For that to work, transfers need to be small enough to actually make
subsequent calls to aquire resources wait. One large call with 1 GB of
data would transfer that 1 GB and make calls afterwards wait, which
might never occur because the 1 GB of data already has been
transferred. From what I've seen in the sources of Axis2 and
HTTPComponents, things should get called already with small chunks of
data because of the usage of "DataHandler".

Not sure if there's an easier way, better ideas welcome. :-)

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-***@axis.apache.org
For additional commands, e-mail: java-dev-***@axis.apache.org

[robertlazarski]

Guten Tag Thorsten Schöning,

So "7.1. Custom client connections" seem to be the way to go: Provide
a custom HttpConnectionFactory and make Axis2 to use that, while I've
already provided a custom connection manager to Axis2 in the past with
non-default settings. What I'm missing currently is the part where the
request body gets serialized, "DefaultHttpRequestWriterFactory" reads
like it only handles headers...
https://hc.apache.org/httpcomponents-client-ga/

tutorial/html/advanced.html#d5e913
The important part to understand seems to be that the connection
factory provides some instances of "HttpClientConnection" in the end
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/
http/HttpClientConnection.html#sendRequestHeader(org.
apache.http.HttpRequest)
https://hc.apache.org/httpcomponents-core-ga/httpcore/apidocs/org/apache/
http/HttpClientConnection.html#sendRequestEntity(org.apache.http.
HttpEntityEnclosingRequest)
The request writers are only used in "sendRequestHeader", while bodies
are serialized using whatever body-containing object is given. In case
of Axis2 this seems to be "AxisRequestEntityImpl", which takes care of
serializing according to some "MessageFormatter", which itself exists
for various different formats like JSON and SOAP and things like MTOM
vs. inline vs. SOAP with attachments are handled somewhere in that
chain as well...
In the end, every data is written to a stream created within
"sendRequestEntity" and that seems to be a good place to put a rate

@Override
public void sendRequestEntity(final HttpEntityEnclosingRequest

request)

throws HttpException, IOException {
Args.notNull(request, "HTTP request");
ensureOpen();
final HttpEntity entity = request.getEntity();
if (entity == null) {
return;
}
final OutputStream outstream = prepareOutput(request);
entity.writeTo(outstream);
outstream.close();
}
protected OutputStream createOutputStream(
final long len,
final SessionOutputBuffer outbuffer) {
if (len == ContentLengthStrategy.CHUNKED) {
return new ChunkedOutputStream(2048, outbuffer);
} else if (len == ContentLengthStrategy.IDENTITY) {
return new IdentityOutputStream(outbuffer);
} else {
return new ContentLengthOutputStream(outbuffer, len);
}
}
protected OutputStream prepareOutput(final HttpMessage message)

throws HttpException {

final long len = this.outgoingContentStrategy.

determineLength(message);

return createOutputStream(len, this.outbuffer);
}

So, the connection factory is needed to provide custom client
connections and one needs to create one which overrides
"createOutputStream" by calling the super implementation and
afterwards wrapping the resulting "OutputStream" once more in some
"RateLimitOutputStream" or such.
From my understanding, that "RateLimitOutputStream" would "only" need
to make sure that writes are somewhat small, so that not e.g. 1 GB of
data is transferred during one call, but smaller blocks of some kB or
MB or such. Each transfer of some small block is forwarded to the
wrapped "OutputStream" after some rate limiting resources have been
https://stackoverflow.com/a/6271935/2055163
For that to work, transfers need to be small enough to actually make
subsequent calls to aquire resources wait. One large call with 1 GB of
data would transfer that 1 GB and make calls afterwards wait, which
might never occur because the 1 GB of data already has been
transferred. From what I've seen in the sources of Axis2 and
HTTPComponents, things should get called already with small chunks of
data because of the usage of "DataHandler".
Not sure if there's an easier way, better ideas welcome. :-)

Its been many years since I used attachments though I was using them for
bit around the time when axis2 first came out.

You seem to be on the right path as your code shows the choices between
using chunked or content-length. And the DataHandler is the other low
hanging fruit place to tweak. MessageFormatter is the heart of the GSON
module that I am using a lot right now and yes, that is another place to
look at customization. .

Since this is http you may have to tweak the timeout.

Large attachments is a subject that comes up occasionally, let us know how
it turns out.

Regards,
Robert

Mit freundlichen GrÌßen,
Thorsten Schöning
--
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - GeschÀftsfÌhrer: Andreas Muchow
---------------------------------------------------------------------

[Thorsten Schöning]

Guten Tag Thorsten Schöning,

Post by Thorsten Schöning
Not sure if there's an easier way, better ideas welcome. :-)

Time to share my experiences: I'm stuck on some old Axis2 1.6.2
currently, which seems to use MultiThreadedHttpConnectionManager,
for which I didn't find how to provide it with a connection factory to
be able to create limiting OutputStreams at all. Maybe I missed
something somewhere...

Nevertheless, at least for my use case I think I have an easier
approach: I need to upload only one file currently and am using SOAP
with attachments to not need to encode Base64 and such. If I
understood correctly, Axis2 demands a DataHandler in those cases and
those can be created based on some DataSource, which itself wraps some
InputStream and that actually wraps the data to upload.

So what I simply did was creating a limiting InputStream around my
data to upload, wrapped that into a DataSource, wrapped that into a
DataHandler and provided that to Axis2. To let Axis2 not consume too
much memory during creation of the request, it needs to be set to use
chunked transfers.[1] In the end, Axis2 is reading somewhat small
packages from my InputStream and writing those directly to the server
and because it can only send what it read, I'm limiting upload
bandwidth that way.

Care needs to be taken regarding the limiter used, I started with
Guava RateLimiter and ran into problems[1] when using it in
Inputstream.read with only reading one byte. Apaches TimedSemaphore
worked better even in those situations. Limiting one byte only instead
of individual packets might be unnecessary, though.

Attaching some PoC in case anyone is interested. Of course it's not
strictly limiting the output, but good enough for what I need
currently. :-)

[1]: http://mail-archives.apache.org/mod_mbox/axis-java-user/201809.mbox/<1486821759.20180923202008%40am-soft.de>
[2]: https://stackoverflow.com/a/52626754/2055163

Mit freundlichen GrÌßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - GeschÀftsfÌhrer: Andreas Muchow