I took another look at diagrams.net. Seems like a well established solution for block diagrams. It also supports a sketch mode so with a graphics tablet, you can easily mix standard shapes and hand sketches (see below for an example).
One thing that is useful is to group objects inside other objects. In draw.io, you can turn any object into a group by selecting arrange->group in the menu – then everything inside is treated as a group. See example below: