With the advancement of education and teaching reform, core literacy has become a new teaching concept that teachers of various subjects must pay attention to. It is mainly student-centered, focusing on students' physical and mental health and all-round development. Physical education has a natural advantage in cultivating the comprehensive quality of college students, and the teaching behavior of college physical education teachers also plays a role that cannot be underestimated. This article studies the behavioral factors of physical education teachers in ordinary colleges and universities in our city. Using the method of factor analysis, the interview and observation method combined with the literature data are used for analysis, and all the physical education teachers in ordinary colleges and universities in our city are interviewed.