Studies on single muscle fiber passive material properties often report relatively large variation in elastic modulus (or normalized stiffness), and it is not clear where this variation arises. This study was designed to determine if the stiffness, normalized to both fiber cross-sectional area and length, is inherently different between types 1 and 2 muscle fibers. Vastus lateralis fibers (n = 93), from ten young men, were mechanically tested using a cumulative stretch-relaxation protocol. SDS-PAGE classified fibers as types 1 or 2. While there was a difference in normalized stiffness between fiber types (p = 0.0019), an unexpected inverse relationship was found between fiber diameter and normalized stiffness (r = −0.64; p < 0.001). As fiber type and diameter are not independent, a one-way analysis of covariance (ANCOVA) including fiber diameter as a covariate was run; this eliminated the effect of fiber type on normalized stiffness (p = 0.1935). To further explore the relationship between fiber size and elastic properties, we tested whether stiffness was linearly related to fiber cross-sectional area, as would be expected for a homogenous material. Passive stiffness was not linearly related to fiber area (p < 0.001), which can occur if single muscle fibers are better represented as composite materials. The rule of mixtures for composite materials was used to explore whether the presence of a stiff perimeter-based fiber component could explain the observed results. The model (R2 = 0.38) predicted a perimeter-based normalized stiffness of 8800 ± 2600 kPa/μm, which is within the range of basement membrane moduli reported in the literature.