Mapping planar structure in vision-based SLAM can increase robustness and significantly improve efficiency of map representation. However, previous systems have implemented planar mapping as an auxiliary process on top of point based mapping, leading to delayed initialisation and increased feature management overhead. We address this by introducing a unified mapping framework based on a common parameterization in which both planar and point features are mapped directly, as and when appropriate according to scene structure. Specifically, no distinction is made between points and planes at initialisation - the 'best' representation emerges after matching has progressed - hence minimizing delay and making the detection of planar structure implicit in the method, avoiding the need for an additional process. We demonstrate the approach within an EKF monocular SLAM system and show its potential for efficient and robust mapping over large areas in both indoor and outdoor environments, including examples of fast relocalisation.