Alberto Cubeddu
1 min readJul 28, 2020

--

1) It could, that's another way to architect the solution, nevertheless, if we are talking about best practice it will break the single-responsibility principle :) One Lambda Function should be responsible for only one task.

2) There is a lot of misconception regarding Kinesis Streams and Shards.

One Stream can have N shards;

Shards can be read at a velocity of 5 reads per second, however, your consumer will read from all the shards;

Therefore if you have 5 lambda consumers you already reached the maximum number of read!

Having more shard, it will help for big batch size item, not to have more consumers.

That's why you have a limitation of 5 consumer per kinesis stream until you don't use the FAN-OUT :)

Regarding your question, i don't know what's your use-case, therefore it's hard to give you a yes/no answer about the bottleneck.

If you are willing to share, we can discuss it :)

3) That's an AWS recommendation/limitation, if you check this page: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html

there is a note saying:

"No more than two processes at most should be reading from the same streams shard at the same time. Having more than two readers per shard can result in throttling."

I don't work for AWS, therefore i can't really know what's behind the scene :)

--

--

Alberto Cubeddu

Leadership || Management || Innovation - Technology Director & Former Head Of Engineering