Adding Video Background to Cocos2d Scene with AVPlayer Layer

Background and Context

Cocos2d is a popular open-source game engine for creating 2D games on various platforms, including iOS, macOS, Windows, and Linux. It provides a comprehensive set of features for building 2D games, including animation, physics, and graphics rendering.

In this blog post, we will discuss how to add a movie in the background of a Cocos2d scene, along with a sprite object above it. We will also explore why the video cannot be moved below the sprite as desired.

Understanding MPMoviePlayerViewController

MPMoviePlayerViewController is a class provided by Apple’s media framework (MPMediaFramework) that allows you to play movies in your iOS application. When you create an instance of this class, you can pass the URL of the movie file you want to play.

In our Cocos2d project, we use MPMoviePlayerViewController to play a movie in the background. Here’s how it works:

  • We first get the path to the movie file using [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"].
  • Then, we create an instance of MPMoviePlayerViewController and pass the URL of the movie file as a parameter.
  • We set various properties on the movie player controller, such as control style, should autoplay, and so on.

Adding the Movie to the Scene

To add the movie to our scene, we use the following code:

- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    MPMoviePlayerViewController *mp = [[MPMoviePlayerViewController alloc] initWithContentURL:movieURL];
    // ... other code ...
}

Adding the Sprite Object

To add a sprite object to our scene, we use the following code:

- (id) init {
    if (self = [super init]) {
        [self startMovie];
        sBall *ball = [sBall Ball];
        CGSize size = [[CCDirector sharedDirector] winSize];
        ball.position = ccp(ball.contentSize.width/2, size.height/2);
        [self addChild:ball z:999];
        [ball moveForever];
        [self beginDetectingTouch];
    }
    return self;
}

Moving the Movie Below the Sprite

Now that we have added both the movie and the sprite object to our scene, let’s see how to make the video appear below the sprite. We can do this by using a layer on top of the sprite.

Here is an example of how you might modify your code:

- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    MPMoviePlayerViewController *mp = [[MPMoviePlayerViewController alloc] initWithContentURL:movieURL];
    // ... other code ...
}
- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    MPMoviePlayerViewController *mp = [[MPMoviePlayerViewController alloc] initWithContentURL:movieURL];
    // ... other code ...
}

Why This Doesn’t Work

Unfortunately, this approach doesn’t work as expected. The reason is that the MPMoviePlayerViewController instance is added to a subview of your OpenGL view’s view (using [[[CCDirector sharedDirector] openGLView]addSubview:mp.moviePlayer.view];) and then set as a sendback using sendSubviewToBack. When you add it, it takes over all the space that you want the video layer to be, so you will not be able to move your video below sprite objects.

We can change this by changing how we add our movie player view.

- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    MPMoviePlayerViewController *mp = [[MPMoviePlayerViewController alloc] initWithContentURL:movieURL];
    // ... other code ...
}
- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    MPMoviePlayerViewController *mp = [[MPMoviePlayerViewController alloc] initWithContentURL:movieURL];
    // ... other code ...
}
- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    MPMoviePlayerViewController *mp = [[MPMoviePlayerViewController alloc] initWithContentURL:movieURL];
    // ... other code ...
}

Final Solution

To fix this, we need to create a custom layer for our video and use that instead of the MPMoviePlayerViewController. Here’s an example of how you might do it:

- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    AVAsset* asset = [[AVAsset alloc] initWithURL:movieURL];
    AVPlayerItem* playerItem = [[AVPlayerItem alloc] initWithAsset:asset];
    self.videoLayer = [[AVPlayerLayer alloc] initWithPlayer:playerItem];
    self.videoLayer.frame = CGRectMake(0, 0, playerItem.asset.duration, playerItem.asset.duration);
    [[[CCDirector sharedDirector] openGLView] addSubLayer:self.videoLayer];
}
- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    AVAsset* asset = [[AVAsset alloc] initWithURL:movieURL];
    AVPlayerItem* playerItem = [[AVPlayerItem alloc] initWithAsset:asset];
    self.videoLayer = [[AVPlayerLayer alloc] initWithPlayer:playerItem];
    self.videoLayer.frame = CGRectMake(0, 0, playerItem.asset.duration, playerItem.asset.duration);
    [[[CCDirector sharedDirector] openGLView] addSubLayer:self.videoLayer];
}
- (void) startMovie {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"sampleVideo" ofType:@"mp4"];
    NSURL *movieURL = [NSURL fileURLWithPath:path];
    AVAsset* asset = [[AVAsset alloc] initWithURL:movieURL];
    AVPlayerItem* playerItem = [[AVPlayerItem alloc] initWithAsset:asset];
    self.videoLayer = [[AVPlayerLayer alloc] initWithPlayer:playerItem];
    self.videoLayer.frame = CGRectMake(0, 0, playerItem.asset.duration, playerItem.asset.duration);
    [[[CCDirector sharedDirector] openGLView] addSubLayer:self.videoLayer];
}

By creating a custom layer for our video and using that instead of the MPMoviePlayerViewController, we can move the movie below the sprite object as desired.


Last modified on 2025-01-21