Name the intended novelty
Decide whether evaluation should test new performers, new recording sessions, unfamiliar productions, new locations, or another source of change. Each question implies different grouping. Randomly assigning clips may preserve shared sets, speakers, branding, or repeated intervals across the boundary. Percentages describe allocation size; they do not establish that the resulting evaluation answers your intended generalization question.
Request relationship metadata
Ask for session, production, participant, location, and original-recording identifiers where relevant and available. Existing archives may not have every relationship documented. Record missingness and its consequences instead of inventing reliable group labels. Keep derivative clips connected to source recordings and alternate edits so a later export does not quietly erase the relationships used to design the split.
Balance with explicit priorities
Keeping groups intact may leave some categories sparse in one split. Decide whether to gather more sources, narrow the evaluation claim, or document a limitation. Do not break the grouping rule silently to obtain attractive counts. Report independent group counts and category coverage together, and review whether a few long productions dominate the evaluation runtime or task examples.
Version the assignments
Freeze split membership with a dataset release and check new acquisitions against existing groups. If a newly discovered source relationship crosses the boundary, document the correction and its effect on comparability. Fiund can include these grouping needs in the sourcing brief, but a supplier’s category or runtime claim should not substitute for evidence that the proposed split is meaningful.
Review checklist
- Define the required source of novelty.
- Request relevant session and production relationships.
- Prioritize grouping constraints over cosmetic balance.
- Version assignments and review new source links.