Recent wide adoption of Electronic Medical Record (EMR) systems provides health practitioners with easy access to patient private information. However, there is a dilemma between the easy access to patient information and the potential privacy infringement brought by such easy access. This paper elaborates three types of group dynamics that identify challenges of privacy management in medical practices: team members, temporal involvement, and different levels of information sensitivity. Drawing on the theory of contextual integrity, this work identifies the appropriate actors, information access, and information transmission principles for understanding the norms of information flows. The findings of the study shed lights on the design insights that privacy enhancing features should be appropriately aligned with the dynamic group behaviors of medical practices.