Without an image, it's a bit hard to visualise what you have.
In this case, I would apply a force every frame on all of the objects that are moving (the player, enemies etc) towards the centre axis of the cylinder.
Looking from the side, it be something like this where the axis of the cylinder is along world Z:
As for keeping the player 'upright' on the cylinder, you have the right idea but this is pretty tricky. You effectively have to construct a transformation matrix with the correct rotation and use the rigidBody.teleport to rotate the rigidBody to the desired rotation.
Getting the correct rotation can be done via calculating the new 'up', 'forward' and 'right' vectors based on the position of the player on the cylinder.
Get the normal of the cylinder either by raycast or by simply getting the difference between the player position and the cylinder axis and normalising it. That will give you the 'up' vector.
Cross the player's 'forward' vector new 'up' vector, that will give you the new 'right' vector.
Cross the new 'right' vector with the new 'up' vector, that will give the inverse of the 'forward' vector which is needed due to a left handed coordinate system.
Then you can construct the transformation mat4 like so:
var transform = new pc.Mat4(
Where x = right vector, y = up vector, z = inverse forward vector.
Now you can get the euler angles from the mat4 transform and pass that to the rigidBody teleport function.
(This is untested and off the top of my head so there may be some errors in my math or some glitches).
If you still have a problem after trying to implementing this, post the project link to the forum with your next issue.