[openamq-dev] PAL examples

[Martin Sustrik]

Carl,

It looks like you've hit the moment when your OS starts swapping. Notice 
that file is read into memory before it is sent (send_file.c).

Let me explain the background here. We are doing a kind of extreme 
programming, i.e. we are not implementing anything that is not used in 
real world. It is not only matter of funding, the reason is also that 
code that is not used tends to turn into crap quite quickly.

So so far requirements from out customers were sending a large amount of 
small messages (up to 20MB) and that's what the system does well.

As for file transfer, it's currently out of focus as we have no customer 
that requires it. To focus on it, we would need real world use cases, 
real world feedback and of course some funding to actually do the work.

With regards to your problem, you can do with splitting the file into 
smaller messages (several MB) and sending those via OpenAMQ. It should 
be fairly simple to implement (adding a simple loop into both send_file 
and receive_file). I can help you with that in case of need.

Martin

Carl Bourne wrote:
> Hi,
> 
> Has there been much performance testing done with OpenAMQ with regards
> to transferring large files?
> 
> Although (with heartbeat disabled) the clients are now able to send
> large files without it disconnecting the clients, the performance seems
> to fairly poor. i.e. about 30 mins to send a 650MB file and about 90
> mins for another client to receive it. The tests done here were done
> locally not even across the network. 
> 
> Whilst the files are transferring it also has a fairly serious hit on
> both CPU and disk IO also.
> 
> Carl
> 
> 
> 
> -----Original Message-----
> From: openamq-dev-bounces at lists.openamq.org
> [mailto:openamq-dev-bounces at lists.openamq.org] On Behalf Of Martin
> Sustrik
> Sent: 30 March 2007 10:26
> To: OpenAMQ development discussion
> Subject: Re: [openamq-dev] PAL examples
> 
> Carl,
> 
>> If you start the sender ( send a file onto the queue ) then start the
>> receiver it doesn't retrieve any files.
>>
>> But if you start the receiver ( waiting for a file ) then start the
>> sender the receiver gets the file.
>>
>> This seems pointless as if your receiver goes down and the senders
> send
>> loads of files you will loss them.
>>
>> Sure there must be a setting somewhere.
> 
> The problem is 'auto-delete' flag on the 
> amq_client_session_queue_declare in receive_file.c
> 
> Auto-delete means that the queue (with all the messages it contains is 
> destroyed when the owning application closes the connection).
> 
> If you want your queue to persist on the broker even if there is no 
> receiver connected, set auto-delete to false.
> 
> Martin
> _______________________________________________
> openamq-dev mailing list
> openamq-dev at lists.openamq.org
> http://lists.openamq.org/mailman/listinfo/openamq-dev
> _______________________________________________
> openamq-dev mailing list
> openamq-dev at lists.openamq.org
> http://lists.openamq.org/mailman/listinfo/openamq-dev