Resource-constrained, wearable and continuous health-monitoring devices require automatic electrocardiogram (ECG) signal quality assessment (SQA) (ECG-SQA) to reduce false alarms and optimize energy consumption. In this paper, the ECG signals are classified as clean and noisy. The performance and computational complexity of convolutional neural network (CNN) based ECG-SQA models are compared for the derivative of ECG (dECG) and Fourier magnitude spectrum of ECG as input. We have evaluated 19 dECG-based and 16 Fourier magnitude spectrum-ECG-based CNNs to find the optimal CNN for ECG-SQA. The CNNs are analyzed for kernel sizes 3 × 1, 4×1, 5 × 1 and 6×1; convolution layers 2, 3 and 4; dense layers 3, 4 and 5; exponential linear unit (ELU) activation function. The proposed CNN-based SQA method with Fourier magnitude spectrum-based ECG-SQA is lightweight and less computationally complex, with a model size of 852 kB and 67,697 parameters. The robustness of the CNN-based ECG-SQA method is tested with two unseen datasets, such as the PhysioNet/Computing in Cardiology Challenge 2011 (PCCC2011) database and the St. Petersburg Institute of Cardiological Technics (INCART) 12-lead arrhythmia database. The method achieved the sensitivity of 99.30% and 98.41%, respectively and specificity of 95.40% and 99.3%, respectively, with unseen PCCC2011 and INCART datasets. For the assessment of 5 s ECG signal, the proposed method has a processing time of 62.6±12.6 ms. © 2023 IEEE.