Objective. Raw electroencephalography (EEG) requires robust rejection of inevitable bad EEG epochs to ensure data reliability. While automated methods reduce manual inspection burdens, existing approaches struggle with parameter optimization, scenario adaptation, and label dependency. This study presents EEGEpochNet, an end-to-end model for accurate bad EEG epoch rejection. Approach. EEGEpochNet is developed through three modules: (1) multi-level morphological representation: A multi-branch 1D-convolutional neural network (CNN) with U-Net-encoded multi-level features captures scale-invariant patterns mimicking expert visual analysis, eliminating handcrafted feature engineering; (2) temporal evolution modeling: bidirectional gated recurrent unit decode electrophysiological dynamics to distinguish artifacts from normal activity; and 3) self-supervised contrastive learning: a symmetric loss leverages unlabeled data to learn domain-invariant EEG representations, reducing reliance on labeled examples. Main results. Extensive experiments have been performed to compare EEGEpochNet to five state-of- the-art counterparts (e.g. Autoreject and BRCNN) on a semi-simulated dataset and two real datasets (the EEG recordings from children and adults): (1) EEGEpochNet performs the best with F1-scores of 93.05%, 95.33%, and 84.41%, and (2) the capability of self-supervised learning makes EEGEpochNet far superior to supervised methods when labeled data are limited. Significance. Overall, EEGEpochNet provides a parameter-efficient framework to deploy reliable EEG analysis toward clinical-grade automation.