In this paper, we consider a problem of parametric identification of a piece-wise linear mechanical system described by ordinary differential equations. We reconstruct the phase space of the investigated system from accelerometer data and perform parameter identification using iteratively reweighted least squares. Two key features of our study are as follows. First, we use a differentiated governing equation containing acceleration and velocity as the main independent variables instead of the conventional governing equation in velocity and position. Second, we modify the iteratively reweighted least squares method by including an auxiliary reclassification step into it. The application of this method allows us to improve the identification accuracy through the elimination of classification errors needed for parameter estimation of piece-wise linear differential equations. Simulation of the Duffing-like chaotic mechanical system and experimental study of an aluminum beam with asymmetric joint show that the proposed approach is more accurate than state-of-the-art solutions.