Ask the Hungarian

TL;DR

1. Unsurprisingly, this question has a mathematical solution based on the Hungarian algorithm.

2. I have not done the calculations, but I imagine that¹ the typical answer is:

   Always ask for feedback from the very beginning. Start with more feedback from the more willing/less skilled beta-readers early on. Save the feedback of less willing and more professional beta-readers for the later stages, when your product is closer to the final form.

_{1: this is based on the guess that the time availability of beta-readers does not scale exponentially relative to their feedback quality ranking. See below for details.}

A far-from-formal discussion of the problem

The setup

you have an assignment problem, with a series of tasks to be split between agents. The task is to provide feedback on a text whose length increases over time, up to an upper limit of which you have a rough estimate (and the accuracy of the estimate increases over time). The agents are a number of beta readers with varying degrees of professionality (i.e. quality of the feedback), patience (i.e. size of the task that they can digest), and available time (i.e. how many times you can ask them to read your writings).

The goal

distribute the workload between beta readers across time to maximize the benefit of your feedback. In math terms, I see this as the maximization of the impact of the feedback, that is the product between the quality of the feedback and the certainty that the text given to the reader is closer to the final draft. It makes no sense to have high-quality feedback on text which you are going to rewrite anyway.

The assumptions

1. You cannot discern your beta-readers based on their taste. You can distinguish them based on their reviewing skills, and on their patience and availability, but they will all like (or hate) the same written works.
2. beta-readers read at the same speed.
3. beta-readers can read unit chunks of text and do not need to read the entire work at once.

Estimating the total length of the text

This total length is the total number of words that you have written. If you never rewrite your work, then it is exactly the number of workds in your novel. If you rewrite parts of it once, then it is the sum of the words in the initial draft, plus the edited parts.

That being said, imagine that your initial writeup goal is 60k words. If we simplify the problem a lot, we can think that for every word there is a certain chance that you will have to rewrite it. Everytime you rewrite a word, the chance of rewriting that specific word decreases. For simplicity, imagine that when you start writing there is a 50% chance that you will rewrite any word. If you rewrite it, there is still a 50% chance that you will rewrite the new one: how many words would have you been writing? 1 word with 50% chance, plus another word with 25% chance, plus another word with 12.5% chance... if you continue forever, and weight the sum by the probability, you get that a 50% rewriting chance results in writing 2 words for any word in the final manuscript: writing a 40k novel will require writing on average 80k words. Obviously, higher rewriting chance require writing more words.

The beta-readers

The beta readers are characterized by:

• the quality of their feedback, which we can treat as a ranking (1: amateurs, 100:professional literary critic)
• the patience: the longest chunk of text that they are willing to read in one go.
• the available time: i.e. the total amount of text that they are willing to read

We can simplify this enormously by splitting each beta-reader in sub-beta-readers, where each sub-reader is simply a unit of time of work of a beta-reader, for instance, a half-hour beta-reading. From the assumption n.2 this corresponds to a fixed amount of words. Now your most willing beta-readers correspond to an army of tiny half-hour subreaders, while your busy beta-readers may only be represented by one sub-beta-reader. From assumption n.3, for the purpose of determining task assignments, there is no distinction between a single beta-reader and an army of sub-beta readers that read the same amount of text. The only practical distinction is that sub-beta-readers cannot read in parallel as they may correspond to the same person. Considering how long it takes to craft a novel, this is an irrelevant detail.

The tasks

Once you have defined your sub-beta-readers, you also know what is the length of the chunk of text that each sub-beta-reader will read: if you have half-hour sub-beta-readers, then it is the hourly reading speed divided by two. How many tasks do you have? Total text length divided by length of a task.

Each chunk of text has a certainty score, which is the inverse of the chance that you will edit it. For instance, if the initial chance of rewriting a word was 50%, then the chunks from the first pass have a certainty score of 2 (1 divided by 50%), the chunks from the first round of editing have a score of 4 (1 divided by 25%), and so forth. This means that the certainty score is an exponential function, i.e. it grows very rapidly with the number of edits.

The measure to optimize

Each sub-beta-reader reads a chunk of your text. The reader has a quality associated to them, given by their relative ranking, as we said above. The chunk of text assigned to the reader has a certainty score. The product of these two numbers is higher for text of which you are more certain and which were reviewed by the more qualified people.

And now what?

Plug all this in the Hungarian algorithm: the agents are the sub-beta readers, and the chunks of text are the tasks. Map the sub-beta-readers back to your beta-readers and the output of the algorithm should tell you which piece of text to give to which reader.

I hope that helps.

Addenda, Notes and PS

• I'm a discovery writer! I don't know how long my novel will be from start!: worry not, you can use the average novel length from your genre as an initial estimate, or you could use a Fermi estimate, or a German tank estimation approach (this may work surprisingly well if you write random scenes to be glued together in the end), or use as upper bound the maximum total length of text that all your beta readers may be willing to read.

• I'm bad at estimating chances of rewriting!: Also, not worry. Start with a guess, like 50%. Then you can apply an expectation-maximization algorithm to tune the chance of rewriting as you actually rewrite. In simple terms, at each round of rewriting, you can plug in the number of words you have actually rewritten and come up with a new guess for what your rewriting chance is. Be careful to track how many times you have rewritten each passage, though.

• What to do if you write more than the number of available readers?: Add pseudo-readers. The ideal case would be to do a random sampling with replacement from your actual beta-readers. A simple and effective way to do that is just to add a bunch of low-quality beta-readers. This will push reading revisions to later stages. Re-run the algorithm if you get new beta-readers.

• What if my readers do not have a similar reading taste?: Group readers in super-readers by taste groups with identical revision quality. Assign chunks of text to each super-reader. Then rerun the algorithm for the group of readers in each super-reader group limiting the text to the collection of chunks assigned to that particular super-reader.

• My readers do not read at the same speed: the conversion to sub-beta readers is done in amount of text per unit of available time. You can also convert a reader to sub-beta readers by the number of text chunks of a given length that they can read. In formulas: avilable_time_for_reader * reading_speed / length_of_basic_text_chunk

• I have already written 40k words! How do I go from here? subtract these 40k words from the total length of the words that need beta-reading. Distribute the work over the remaining text.

• When should I run this algorithm? You can run it even before you start writing. You can re-run it everytime a beta-reader reads a chunk of your text: you remove that amount of text from the total text to be read, remove the corresponding sub-beta-reader, update the estimated total length and the chance of rewriting, and rerun the algorithm. Doing so will allow you to distribute your work more precisely and in a more timely fashion. Once you have it, it is probably a matter of a couple of click.

• I want to give my reader single chapters, rather than chunks of text, and they are of different length: as long as the readers will consider all chapters to be equal in terms of workload, replace the text chunks with chapters in the computation above. Each reader is divided in sub-readers corresponding to the total number of chapters that they are willing to read. Your novel length is given in chapters, and the probability of rewriting, refers to a chapter and not to a single word anymore. Leave the rest unchanged. A word of advice, do not disclose whether a chapter is longer or shorter than average to your beta-readers unless you are sure that they are not easily offended by workload inequality.

• Is there readily available software to calculate this? yes and no, and sadly the margin is too narrow for me to put it here :)

posted over 5 years ago

CC BY-SA 4.0

_X_‭

420 reputation 20 121 65 1

I'd say that you would do well to compile a list for yourself of the type of criticism that you are looking for. Because what you'll get, if you throw your work out to readers without a specific task for the reader, is everything ranging from people deciding not to read it alt all, people saying 'Looks great!' (and you suspect they didn't read it either), people pointing out comma errors and tense mistakes and other minor things that really don't help much in terms of story, people identifying all the technical errors you are blind to or don't know yet, like repeating words / telling instead of showing / viewpoint issues, or people rewriting your work wholesale, or people saying your characters have no goals, your story no arc, your scenes no structure...

In other words, there's a good chance what you'll be given in return is not what you expect if you don't constrain the request up front. (Also, spend time identifying good readers. These are usually writers.)

So. Compile a list of the feedback you want, at any stage of writing that you desire. Compiling the list will also help you find mistakes on your own. Not sure what to put on the list? A good place to get ideas is on writing blogs, and also by spending time critiquing other writers' work.

I recently took a single sheet to a group I'm in, and on this sheet I had story-boarded a trilogy I'd like to write. I wanted constructive feedback on the ideas, and we spent about thirty minutes hashing through the story arcs I had sketched for the trilogy. There was not a single named character yet, no words at all in story, but I wanted feedback on the concepts.

Answer: So, at any stage in the story writing process, you can solicit constructive feedback--but you probably are doing yourself a disservice if you don't know the specific type of feedback you want beforehand. (Most beginning writers say "Is it interesting?" and I'd urge you to look a little deeper than that, because having been on the receiving end of that a number of times, I've come to realize whether it is interesting or not still boils down to details of crafting a story.)

posted over 5 years ago

CC BY-SA 3.0

DPT‭

660 reputation 40 73 114 0

at what point to ask for constructive critisism.

It depends on how you write. Some people plot out their novels in great detail, every foreshadowing, plot turn and twist. They know their characters backward and forward, and everything that will change about them during the story. So when they are done with Chapter One, and have finished X drafts of it, Chapter One is done forever. If that is how you write, then you might as well get feedback on each Chapter as you finish it.

I don't write like that, I am a discovery writer. I typically write at least two openings, in my current story I wrote four. I invent my plot as I go, my additional characters, I may delete them or replace them. I begin with a protagonist I know better than others, but even she may get some personality changes, I'll find out when I put her in new situations.

So I don't seek feedback until I am DONE with the story. Because until then, I might change any part of it! Fortunately, I have friends that think I write well, they know I can finish a novel and it will feel like a novel, and they don't mind reading the whole thing in their spare time.

I will also note that the most important parts you need reviewed are the opening (first 10 pages, first 25 pages) and the climax near the end (the resolution of the protagonist's major problem, which is not necessarily in the last chapter). You don't need detailed critique of the middle, though you might get it. Listen to it.

About asking for constructive criticism: I tell my readers I am looking for negative feedback much more than positive feedback, the whole point is to help me find problems I was too close to see, or too dumb to realize. So no hurt feelings on my part, I want a story that works more than I want any kind of polite praise. I want them to know the next step is sending it to a stranger, likely a brutal agent that rejects 90% of what she reads. So we want to fix any problems before that happens, not learn about them only after I've burned my best prospects.

That is how people that don't want to hurt your feelings actually hurt your chances of success instead.

On answering criticism: I don't reject it, even if they are completely, totally, utterly wrong and don't get it. Either they are right, or I will look at it and see if I can find a solution.

It is hard enough for people to tell me honestly they don't like what I wrote, without getting punished for it by me calling them an ignorant slut reading at the grade level of a Labrador retriever.

Sometimes their own morality or politics conflicts with mine, sometimes their criticism arises from my failure to communicate clearly. Inconsistency is a problem to solve, poor communication is, but a difference in morality or politics or world view is not. Let 'em rant, don't make excuses or try to explain what you meant, don't get in an argument.

posted over 5 years ago

CC BY-SA 4.0

Amadeus‭

3156 reputation 2 1399 613 37