Fake speech detection is essential to defend speech spoofing attacks of imitation through Artificial Intelligence Generated Content (AIGC) technologies including text-to-speech synthesis (TTS) and voice conversion (VC). Existing solutions are classification-based and rely on large data, showing limitations in solving both data diversity and result explainabilty problems. To resolve these limitations, we design a Brain-inspired Multi-Detector Machine (BiMDM), which is inspired by the brain’s perception and decision-making mechanisms. Our method proposes to use multiple detectors to capture various aspects of fake speech characteristics. To ensure the final detection precision, each detector is trained with the aim of Maximum Detection Precision (MDP) for a specific forgery clue, unlike previous classifiers optimized for Minimum Classification Error (MCE). And a sufficient number of detectors are necessary to reduce the total detection miss rate. This mechanism assigns meaningful roles to each individual detector as well as ensures the detection performance. Then the detectors’ results are integrated through an overall explainable decision-making module, including OR logic calculus and decision trees, to produce result with explainability of the entire detection process. Our experimental results demonstrate the effectiveness of our multi-detector machine and reveal the potential of our proposed novel perspective for fake speech detection task.