Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom video encoder settings and dynamic screen sizes #406

Closed
wants to merge 12 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
12 changes: 12 additions & 0 deletions SmartDeviceLink-iOS/SmartDeviceLink/SDLStreamingMediaManager.h
Expand Up @@ -82,9 +82,21 @@ typedef void (^SDLStreamingStartBlock)(BOOL success, NSError *__nullable error);
*/
- (BOOL)sendAudioData:(NSData *)pcmAudioData;

/**
* The settings used in a VTCompressionSessionRef encoder. These will be verified when the video stream is started. Acceptable properties for this are located in VTCompressionProperties. If set to nil, the defaultVideoEncoderSettings will be used.
*
* @warning Video streaming must not be connected to update the encoder properties. If it is running, issue a stopVideoSession before updating.
*/
@property (strong, nonatomic, nullable) NSDictionary* videoEncoderSettings;

@property (assign, nonatomic, readonly) BOOL videoSessionConnected;
@property (assign, nonatomic, readonly) BOOL audioSessionConnected;

/**
* Provides default video encoder settings used.
*/
@property (strong, nonatomic, readonly) NSDictionary* defaultVideoEncoderSettings;


@end

Expand Down
115 changes: 53 additions & 62 deletions SmartDeviceLink-iOS/SmartDeviceLink/SDLStreamingMediaManager.m
Expand Up @@ -127,6 +127,28 @@ - (BOOL)sendAudioData:(NSData *)pcmAudioData {
return YES;
}

#pragma mark - Update video encoder

- (void)setVideoEncoderSettings:( NSDictionary * _Nullable)videoEncoderSettings {
if (self.videoSessionConnected) {
@throw [NSException exceptionWithName:SDLErrorDomainStreamingMediaVideo reason:@"Cannot update video encoder settings while video session is connected." userInfo:nil];
return;
}

_videoEncoderSettings = videoEncoderSettings;
}

- (NSDictionary*)defaultVideoEncoderSettings {
static NSDictionary* defaultVideoEncoderSettings = nil;
if (defaultVideoEncoderSettings == nil) {
defaultVideoEncoderSettings = @{
(__bridge NSString*)kVTCompressionPropertyKey_ProfileLevel : (__bridge NSString*)kVTProfileLevel_H264_Baseline_AutoLevel,
(__bridge NSString*)kVTCompressionPropertyKey_RealTime : @(YES)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@(YES) can be written as @YES

};
}
return defaultVideoEncoderSettings;
}


#pragma mark - SDLProtocolListener Methods

Expand Down Expand Up @@ -229,83 +251,52 @@ - (BOOL)sdl_configureVideoEncoderWithError:(NSError *__autoreleasing *)error {

if (status != noErr) {
// TODO: Log the error
if (*error != nil) {
if (*error == nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionCreationFailure userInfo:@{ @"OSStatus" : @(status) }];
}

return NO;
}

// Set the bitrate of our video compression
int bitRate = 5000;
CFNumberRef bitRateNumRef = CFNumberCreate(NULL, kCFNumberSInt32Type, &bitRate);
if (bitRateNumRef == NULL) {
// TODO: Log & end session
if (*error != nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationAllocationFailure userInfo:nil];
}

return NO;

if (self.videoEncoderSettings == nil) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is how it's being done, then the video encoder settings dict should be null_resettable, not nullable, and the api should be redesigned so that it's initialized on init, maybe an initWithEncoderSettings if possible. For testability, as many dependencies as possible should be injected at init time.

Copy link
Contributor Author

@asm09fsu asm09fsu May 27, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initWithEncoderSettings: would collide with other fixes coming down the pipeline (such as dynamic encoder dimensions). The thought here is that so long as the stream isn't open, we can modify the existing SDLStreamingMediaManager object, instead of having to specify it via the proxy (since the developer never calls the init). We can definitely add it in the init.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to decide how this init will work with #400 also making a change / addition to init.

self.videoEncoderSettings = self.defaultVideoEncoderSettings;
}

status = VTSessionSetProperty(self.compressionSession, kVTCompressionPropertyKey_AverageBitRate, bitRateNumRef);

// Release our bitrate number
CFRelease(bitRateNumRef);
bitRateNumRef = NULL;

// Validate that the video encoder properties are valid.
CFDictionaryRef supportedProperties;
status = VTSessionCopySupportedPropertyDictionary(self.compressionSession, &supportedProperties);
if (status != noErr) {
// TODO: Log & End session
if (*error != nil) {
if (*error == nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionSetPropertyFailure userInfo:@{ @"OSStatus" : @(status) }];
}

return NO;
}

// Set the profile level of the video stream
status = VTSessionSetProperty(self.compressionSession, kVTCompressionPropertyKey_ProfileLevel, kVTProfileLevel_H264_Baseline_AutoLevel);
if (status != noErr) {
if (*error != nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionSetPropertyFailure userInfo:@{ @"OSStatus" : @(status) }];
}

return NO;
}

// Set the session to compress in real time
status = VTSessionSetProperty(self.compressionSession, kVTCompressionPropertyKey_RealTime, kCFBooleanTrue);
if (status != noErr) {
if (*error != nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionSetPropertyFailure userInfo:@{ @"OSStatus" : @(status) }];
}

return NO;
}

// Set the key-frame interval
// TODO: This may be unnecessary, can the encoder do a better job than us?
int interval = 50;
CFNumberRef intervalNumRef = CFNumberCreate(NULL, kCFNumberSInt32Type, &interval);
if (intervalNumRef == NULL) {
if (*error != nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationAllocationFailure userInfo:nil];

for (NSString* key in self.videoEncoderSettings.allKeys) {
if (CFDictionaryContainsKey(supportedProperties, (__bridge CFStringRef)key) == false) {
if (*error == nil) {
NSString* description = [NSString stringWithFormat:@"\"%@\" is not a supported key.", key];
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionSetPropertyFailure userInfo:@{NSLocalizedDescriptionKey : description}];
}
CFRelease(supportedProperties);
return NO;
}

return NO;
}

status = VTSessionSetProperty(self.compressionSession, kVTCompressionPropertyKey_MaxKeyFrameInterval, intervalNumRef);

CFRelease(intervalNumRef);
intervalNumRef = NULL;

if (status != noErr) {
if (*error != nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionSetPropertyFailure userInfo:@{ @"OSStatus" : @(status) }];
CFRelease(supportedProperties);

// Populate the video encoder settings from provided dictionary.
for (NSString* key in self.videoEncoderSettings.allKeys) {
id value = self.videoEncoderSettings[key];

status = VTSessionSetProperty(self.compressionSession, (__bridge CFStringRef)key, (__bridge CFTypeRef)value);
if (status != noErr) {
if (*error == nil) {
*error = [NSError errorWithDomain:SDLErrorDomainStreamingMediaVideo code:SDLStreamingVideoErrorConfigurationCompressionSessionSetPropertyFailure userInfo:@{ @"OSStatus" : @(status) }];
}

return NO;
}

return NO;
}

return YES;
Expand Down