Dissipationless microwave radio frequency superconducting interference device (RF-SQUID) multiplexers are attractive for readouts of large format arrays of transition edge sensors (TES). This multiplexer has the potential for readout bandwidth of several GHz and power consumption of ≈0.1 pW per pixel, which are ≈103 and ≈10-4 times as large as those of DC-SQUID-based conventional multiplexers. To evaluate their readout performance for TES photon counters ranging from near infrared to gamma ray, white noise of this multiplexer was experimentally studied at 4 K. The reason for this was, firstly, to avoid the low-frequency fluctuation obvious at around 0.1 K but trivial for the energy resolution of photon counters, and secondly for a feasibility study of readout operation at 4 K for extended applications. To increase resonant Q at 4 K and maintain low-noise SQUID operation, multiplexer chips consisting of niobium nitride (NbN)-based coplanar-waveguide resonators and niobium (Nb)-based RF-SQUIDs have been developed. This hybrid multiplexer exhibited 1×104≤QI≤2×104 and the square root of spectral density of current noise referred to the SQUID input √SI =31 pA/√Hz, where QI is the internal Q of resonators. The former and the latter are factor-of-five and seven improvements from our previous results on Nb-based resonators, respectively. Systematic noise measurement with various microwave readout power PMR and two-directional readout on the complex plane of transmission component of scattering matrix S21 makes the contribution of each noise source obvious. By decreasing these noises to the degree achievable by current technology, we predicted the microwave RF-SQUID multiplexer would exhibit √SI ≤5 pA/√Hz; i.e., close to √SI of state-of-the-art DC-SQUID-based multiplexers.