The “only build pull request” setting in Project Settings > Advanced has been updated to clarify that it only applies to OAuth pipelines, not GitHub App pipelines.
This is an important clarification, given that organizations that use the GitHub OAuth App can now also run pipelines integrated via GitHub App to leverage new features.