A few more thoughts:
Pre-screening in the colony would take a fair amount reading time, which personally I don’t have. Yes, I can find the time to watch pop ups but I wouldn’t' be able to spare time for pre-screening the subs. I work. I already have to restrict myself if / when I give feedback to other writers. Then I have my own writing. I wouldn’t be able to read 5-6 subs every week and give a thumbs up or down. My own personal reading / research is already taking a back seat.
Another question: how many colony members would be willing and able to take the time to pre-screen? You might end up with only one or two voices on some occasion. I’m sure they would be good voices, but the process is so subjective. They may turn something down which you would think is fab but never get to see.
Also, what if a submitter isn’t happy that their sub didn’t make it past the colony, where does that leave the colony? Do you want to burden the colony with the Karma of the submission process, and the potential fall out? I think the ‘no’s should come from you, because the 'yes's will do.
If I look at it from a submitters POV, I would want the industry input, not the colony’s; I would want your input, Pete’s input. It’s not that I don’t trust the colonists, I do, totally, entirely, but if I send a sub, I’d like it to end up in front of someone who navigates the industry, someone in your seat, the selling seat, the man who talks to the publishers. To me, the colony input is what happens in the writing groups.
And there will be submitters who aren’t colony members. They might not be that happy that the colony decides whether or not their sub goes through to the pop up and in front of the panel. They may be a bit miffed that
@AgentPete wasn’t the one to make the decision, especially since they invested years on their work.
One thing that concerns me
most about this is that a pre-screening would create a gatekeeper culture in the submission process. Pass the colony test and you will be put in front of the ‘big man’. Is this the way you want to go?
The thing is, if you decide to take someone on as a client following the pop ups, I feel it should be your process all the way up.