Part 6

11.01.2017
Part6
InternetEngineering
Dr.MarekWoda
MultimediaandComputerVisualisation
MPEG-2compression
MovingPicturesExpertsGroup(MPEG)- 1988
• ISO- InternationalStandardOrganisation
• CCITT- ComitéConsultatifInternational deTéléphonieetTélégraphie
• CCIR- ComitéConsultatifdelaRadiodiffusion
Thefirstreportonthework (MPEG-1)- 1990
Thefirstdemonstrationofresults- 1991
CompletereportwithadescriptionoftheMPEG-2- 1994
• MPEG-1- codingsequenceofimagesandsoundsto
writeforthecompactdiscsandvideo
• MPEG-2- codingsequenceofimagesandsound
primarilyfortelevision
Theneedforcompression- asimpleestimate:
oneimage=2x(144+720)samplesx625lines=1.080.000samples
onesample=8bits=1byte(B)
VANCField1
oneimge(frame)=1.080.000B=1,03MB
onesecondmovie=1,03MBx25images
=25.75MB
HANC
oneCD(700MB)=700/25.75≈27sof
a movie
onemovie(1.5h=5400s)=5400/27≈
200CDs
2x144
samples
24lines
Field1
288lines
VANCField2
25lines
Field2
288lines
2x720samples
(720Y+2*360U,V)
6
1
11.01.2017
MPEG-2standard- themostimportantdocuments:
MPEG-2standard- themostimportantdocuments (continued):
ISO/TEC DIS 13818-1 - SYSTEM - synchronization and switching
of video and audio signals
ISO/TEC DIS 13818-5 - SIMULATION - a technical description of
the software implementation of the first three parts of the
standard.
ISO/TEC DIS 13818-2 – VIDEO - encoding the video signal
(interlace and progressive scan)
ISO/TEC DIS 13818-3 – AUDIO - coding of audio signals (an
extension for encoding multi-channel, surround sound, sounds,
multilingual)
ISO/TEC DIS 13818-4 – COMPARATIVE TESTS - determination of
the characteristics of the encoding and decoding and to test the
compatibility with other parts of the standard
MPEG-2Video- VideoStream DataHierarchy
ISO/TEC DIS 13818-6 - EXTENSION SYSTEM - Command and
control of digital media processing, describes the protocol for
the client and server applications
ISO/TEC DIS 13818-9 – INTERFACE SPECIFICATION FOR THE
APPLICATION PROCCESING, HIGH QUALITY, REAL TIME - defines
the exact timing for the transmission of MPEG-2 video in real
time.
+ additional documents
MPEG-2generalideaofcompressionalgorithm
2
11.01.2017
MPEG-2Video- VideoStream DataHierarchy
Ageneral algorithm forimagesequence compression
1. Sampling andquantization consecutive imageframes
2. Coding ofimageframes
frames
1.Samplingandquantizationoftheimageframes
Sampling and
quantsation
frames
bitstream
Coding offrames
Chromasubsampling
Result of sampling and quantization of picture frame: Three
tables describing the image in the luminance – chrominance
model ( YIQ , YUV )
Theyareusedfollowingsamplingschemes:
4:4:4
4:2:2
4:2:0
4:1:1
próbki
luminancji
luminace
sample
próbki
chrominancji
chrominance
sample
3
11.01.2017
Typicalsizesoftablesfortheframe (PALsystem625,lines)
4:4:4
4:4:4
4:2:2
4:2:0
4:1:1
4:1:1
Y
U
720 x 576
V
720 x 576
88bit/point
bitów/punkt
8bit/point
8
bitów/punkt
Y
U
V
360 x 576
88bit/point
bitów/punkt
8bit/point
8
bitów/punkt
Y
U
V
360 x 288
88bit/point
bitów/punkt
8bit/point
8
bitów/punkt
Y
U
V
720 x 576
180 x 576
88bit/point
bitów/punkt
luminancja
TV
360 x 288
88bit/point
bitów/punkt
88bit/point
bitów/punkt
Codedusingonlyinformationpresentinthepictureitself,andprovidespotentialrandomaccess
pointsintothecompressedvideodata.Itusesonlytransformcodingandprovidemoderate
compression.Typicallyitusesabouttwobitspercodedpixel.
360 x 576
88bit/point
bitów/punkt
720 x 576
Intraframes
720 x 576
88bit/point
bitów/punkt
720 x 576
Typesofimageframes
TV
180 x 576
chrominancja
chrominancja
2.1.1.I (Intra)frames - encoding algorithm
2.1.Determinationoftheorderencoding
PP
BB
BB
PP
GOP- Group ofPictures
• Iframe (Intra)
• Pframe (Predicted)
• Bframe (Bidirectional)
BB
TypeIframeisencodedusingtheJPEGalgorithm.
Thisisthemostaccuratemethod toframeencodingalgorithm
usedinMPEG-2.
JPEG- themostimportantsteps
Exampleofcodingorderofframes
BB
Bidirectionalframes
8bit/point
8
bitów/punkt
• Determinationoftheorderencoding
• Encodingofconsecutiveframesaccordingtoaspecificorder
BB
CodedwithrespecttothenearestpreviousI- orP-frames.Thistechniqueiscalledforward
prediction.LikeI-pictures,P-framesalsocanserveasapredictionreferenceforB-framesand
futureP-frames.Moreover,P-framesusemotioncompensationtoprovidemorecompression
thanispossiblewithI-pictures.
Bidirectionalframes,arepicturesthatusebothapastandfuturepictureasareference.This
techniqueiscalledbidirectionalprediction.B-framesprovidethemostcompressionsinceituse
thepastandfuturepictureasareference,however,thecomputationtimeisthelargest.
2.Coding ofimageframes
... II
Predictedframes
BB
II ...
•
•
•
•
•
divisiontheimageintoblocks(8x8pixels),
calculatingtheDiscreteCosineTransform(DCT)forblocks
quantizationofDCTcoefficients,
scanningofDCTcoefficientstablesusingzig-zagalgorithm
entropycoding
4
11.01.2017
2.1.1.I (Intra)frames - encoding algorithm
2.1.2.P (Predicted)frames - encoding algorithm
Videoencodingusingmotionvectors.
Assumptions:
•framesweredividedintosmallerparts(macroblocks),
•macroblockswillbefurthertreatedindependently,
•frame"close"toeachotherarequitesimilar.
2.1.2.P (Predicted)frames - encoding algorithm
Motionvector- thebasicrule:
encoded macroblock
makroblok
(x,y) kodowany
mkod
ramka
odniesienia
reference
frame
mencod
(x, y)
(xo, yo)
[u, v]= [xo- x, yo- y]
motion
wektorvector
ruchu
mo0
m
reference macroblock
makroblok
odniesienia
Ifm0 is"similar"tomncod,approximately mncod canbewrittenas:
m encod (i , j ) » m o (i + u , j + v )
5
11.01.2017
So, to encode a macroblock mencod assuming that the reference
frame is recorded, is sufficient to provide only the motion vector
[u, v].
Applicationofthisprincipleinpractice,however,requires
answeringseveralquestions.
• Whatsizeshouldhaveamacroblock?
• Whichframeisaframeofreference?Previous?Next?
• Whatismeantby"similar"macroblock?
• Whatisthereferenceframelook"similar"macroblock?
• Whattodowhenthereisno"similar"macroblock?
Motionvectorcalculation(eg. algorithm)
(x-64, y-64)
coded macroblock
makroblok
(x,y) kodowany
(x-64, y+63)
mkod (16 x 16)
(x,y)
(x+63, y-64)
• Macroblock size is 16 x 16 pixels (standard MPEG-2).
• Frame of reference is the closest previous frame type I, or P
(standard MPEG-2).
• What is meant by "similar" macroblock, and what area it
sought, MPEG-2 standard does not specify. Motion vector
search method [u, v] in this case depends on the
implementation.
• If the motion vector search method allows negative result, the
macro block is encoded using the JPEG algorithm (MPEG-2
standard).
d (u , v ) =
ramka kodowana
ramka odniesienia
P (Predicted)frames - encoding algorithm (contd.)
(x+63, y+63)
obszar area
poszukiwania
search
Inthesearchareaexaminesallpossiblemacroblocklocationof
canandchoosestheonethatminimizesthecriteriond (u, v).
where:
1
256
15
15
åå
i =0 j =0
m kod ( x + i , y + j ) - m o ( x + i + u , y + j + v )
- 64 £ u , v £ 63
Motionvector[u, v] willbealwaysfound.
Directsearchsolution[u, v] foronemacroblockrequiresto
calculatethevalueofthefunctiond (u, v), asmanytimesas
pixelcontainsthesearcharea (16,384times).
Countingthevalueof d (u, v) mustperformafewhundred
arithmeticoperations.
6
11.01.2017
P (Predicted)frames - full encoding algorithm
2.1.3.B (Bidirectional)frames - encoding algorithm
1. Divide thecodedframeintomacroblocks.
2. Usingaframeofreferencedeterminemotionvectorsfor
macroblocks.
3. Forcodedframe,usingmotionvectors, calculateencode
predictionerror.
TypeB frameisencodedusingnext,previousandnextframeof
typeI or P.
• Basedonthereferenceframeandmotionvectors,
reconstructcodedframe.
• Calculatethedifferencebetweensourceframeand
reconstructedframe- theresultistheso-called
predictionerror.
• EncodepredictionerrorusingtheJPEGalgorithm
(differentquantizationtable).
Calculates twomotionvectors[u1, v1] and[u2, v2].
ramka
odniesienia
reference
frame 2 2
T2
ramka kodowana
encoded
frame
T1
encoded macroblock
makroblok
(x,y) kodowany
ramka
odniesienia
reference
frame 1 1
mkod
m
enc
d
(x, y)
...
T2
T1
I
B
T1
B
T2
T2
T1
B
P
T1
time
B
P
...
T2
ForthedifferentframesoftypeB, T1 andT2 mayvary.
Calculates twomotionvectors[u1, v1] and[u2, v2].
(x2, y2)
(x, y)
czas
Forexample:
m22
m
[u2, v2]= [x2 - x, y2 - y]
makroblokmacroblock
odniesienia
reference
2
(x1, y1)
[u1, v1]= [x1 - x, y1 - y]
mm11
reference
makroblokmacroblock
odniesienia
1
m encod (i , j ) »
T2
T1
m1 (i + u1 , j + v1 ) +
m 2 (i + u2 , j + v 2 )
T1 + T2
T1 + T2
7
11.01.2017
B (Bidirectional)frames - full encoding algorithm
3.Controlbitratedatastream
1. Divide thecodedframes intomacroblocks.
2. Usingaframes ofreferencedeterminemotionvectors for
macroblocks.
3. Forcodedframe,usingmotionvectors, calculateencode
predictionerror.
• Basedonthereferenceframes andmotionvectors,
reconstructcodedframe.
• Calculatethedifferencebetweensourceframeand
reconstructedframe- theresultistheso-called
predictionerror.
• EncodepredictionerrorusingtheJPEGalgorithm
(differentquantizationtable).
Controlof
quantization
tables
sequence
offrames
Encoder
I,P,B
Output buffer
bitstream
(constant bitrate )
Byexaminingthestateoftheoutputbuffer,youcanchangethe
quantizationtablesinordertoachieveaconstantrateforthe
outputbitstream.
3.Codingsequenceofframes(codingorder)
3.Coding sequence offrames (coding order)
Coding
...
I
B
B
P
B
B
P
1
2
3
4
5
6
7
...
Recording,transmission,decoding:
...
I
P
B
B
P
B
B
1
4
2
3
7
5
6
...
8
11.01.2017
5.LimitationsoftheMPEG-2system
Le v e l
s a m ple /
r ow
r ow s /s e c .
H igh
1920
H igh 1 4 4 0
DTV - coder and transmitter block diagram
fra m e s /
sec.
s a m ple s
( lu m ina n c e )
/s e c .
M b /s
1152
60
6 2 . 6 6 8. 80 0
80
1440
1152
60
4 7 . 0 0 1. 60 0
60
M a in
720
576
30
1 0 . 3 6 8. 00 0
15
Lo w
352
288
30
3.041.280
4
video
Coding
and
compression
Transport
audio
Coding
Channel
coding
Modulation
Multiplekser
and
compression
controldata
additionaldata
DVB-S
DVB-T
MPEG-2
9