I made good on what I said yesterday: I scrapped all those fancy interfaces that I had and merged them all into Filter. The resulting simplification of some parts of the code was noticable, the commit added 681 lines but removed 906.
I introduced a small metadata handling bug that I will take care of right after finishing this post and I still need to create a filter that can contain several filters so that I can simply attach a pipeline (consisting of e. g. a stream source and an mp3 decoder) to a multi source filter. That should fix all my format-metadata-changing problems.