US20060218482A1 - Mobile imaging application, device architecture, service platform architecture and services - Google Patents
Mobile imaging application, device architecture, service platform architecture and services Download PDFInfo
- Publication number
- US20060218482A1 US20060218482A1 US11/357,661 US35766106A US2006218482A1 US 20060218482 A1 US20060218482 A1 US 20060218482A1 US 35766106 A US35766106 A US 35766106A US 2006218482 A1 US2006218482 A1 US 2006218482A1
- Authority
- US
- United States
- Prior art keywords
- video
- mobile
- present
- handset
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/022—Electronic editing of analogue information signals, e.g. audio or video signals
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
- H03M7/4006—Conversion to or from arithmetic code
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/34—Network arrangements or protocols for supporting network services or applications involving the movement of software or configuration parameters
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
- H04N1/00281—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a telecommunication apparatus, e.g. a switched network of teleprinters for the distribution of text-based information, a selective call terminal
- H04N1/00307—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture with a telecommunication apparatus, e.g. a switched network of teleprinters for the distribution of text-based information, a selective call terminal with a mobile telephone apparatus
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00962—Input arrangements for operating instructions or parameters, e.g. updating internal software
- H04N1/00973—Input arrangements for operating instructions or parameters, e.g. updating internal software from a remote device, e.g. receiving via the internet instructions input to a computer terminal
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
- H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/164—Feedback from the receiver or from the transmission channel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/164—Feedback from the receiver or from the transmission channel
- H04N19/166—Feedback from the receiver or from the transmission channel concerning the amount of transmission errors, e.g. bit error rate [BER]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/167—Position within a video image, e.g. region of interest [ROI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/1887—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a variable length codeword
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/34—Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/62—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding by frequency transforming in three dimensions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
- H04N19/635—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets characterised by filter definition or implementation details
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/93—Run-length coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
- H04N21/43637—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
- H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/61—Network physical structure; Signal processing
- H04N21/6106—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
- H04N21/6131—Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via a mobile phone network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/61—Network physical structure; Signal processing
- H04N21/6156—Network physical structure; Signal processing specially adapted to the upstream path of the transmission network
- H04N21/6181—Network physical structure; Signal processing specially adapted to the upstream path of the transmission network involving transmission via a mobile phone network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N2201/00—Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
- H04N2201/0077—Types of the still picture apparatus
- H04N2201/0084—Digital still camera
Definitions
- Directly digitized images and video take lots of bits; it is common to compress images and video for storage, transmission, and other uses.
- Several basic methods of compression are known, and very many specific variants of these.
- a general method can be characterized by a three-stage process: transform, quantize, and entropy-code.
- Most image and video compressors share this basic architecture, with variations.
- the intent of the transform stage in a video compressor is to gather the energy or information of the source picture into as compact a form as possible by taking advantage of local similarities and patterns in the picture or sequence.
- No compressor can possibly compress all possible inputs; we design compressors to work well on “typical” inputs and ignore their failure to compress “random” or “pathological” inputs.
- Many image compression and video compression methods such as MPEG-2 and MPEG-4, use the discrete cosine transform (DCT) as the transform stage.
- DCT discrete cosine transform
- Quantization discards information after the transform stage; the reconstructed decompressed image cannot then be an exact reproduction of the original.
- Entropy coding is generally a lossless step: this step takes the information remaining after quantization and codes it so that it can be reproduced exactly in the decoder. Thus the design decisions about what information to discard is not affected by the following entropy-coding stage.
- DCT-based video compression/decompression (codec) techniques A limitation of DCT-based video compression/decompression (codec) techniques is that, having been developed originally for video broadcast and streaming applications, they rely on the encoding of video content in a studio environment, where high-complexity encoders can be run on computer workstations. Such computationally complex encoders allow computationally simple and relatively inexpensive decoders (players) to be installed in consumer playback devices.
- players computationally simple and relatively inexpensive decoders
- asymmetric encode/decode technologies are a poor match to mobile multimedia devices, in which video messages must be captured in real time in the handset itself, as well as played back.
- video in mobile devices is typically limited to much smaller sizes and much lower frame rates than in other consumer products.
- This invention pertains to methods, devices, systems, and architectures relating to still image and video image recording in devices, including mobile devices, corresponding mobile device architectures, service platform architectures and methods and services for transmitting, storing, editing, sharing, marketing, and transcoding still images and video images over wireless and wired networks and systems and viewing them on display-enabled devices, as well as network and other system services in relation to the foregoing.
- the present invention also pertains to improvements in the image recording technique, and corresponding improvements in the architectures of mobile devices and service platforms.
- aspects of the present invention comprise all-software video codecs/camcorder applications for compressing and/or decompressing video or still images.
- aspects of the present invention also comprise infrastructure products, methods and processes, including mobile multimedia service (MMS) infrastructure applications, for deploying video messaging and sharing services in conjunction with software video codec/camcorder applications for mobile handsets as well as editing and transcoding applications to support complete interoperability with other commonly-deployed standards-based and proprietary video formats.
- aspects of the invention also comprise methods, processes and business processes for establishing, enabling, distributing and operating innovative MMS services, including an innovative mobile video blog and marketing service for video content created by mobile users on mobile devices.
- FIG. 1 depicts Video image size limitations in mobile image messaging.
- FIG. 2 depicts a system diagram for joint source-channel coding: (a) encoder; (b) decoder.
- FIG. 3 depicts a Mobile Imaging Handset Architecture.
- FIG. 4 depicts a Mobile Imaging Service Platform Architecture.
- FIG. 5 depicts a Video Codec Technology Comparison.
- FIG. 6 depicts a system diagram for improved joint source-channel coding: (a) encoder; (b) decoder.
- FIG. 7 depicts an improved mobile imaging handset platform architecture.
- FIG. 8 depicts a Video Codec Performance Comparison.
- FIG. 9 depicts an improved mobile imaging handset platform architecture.
- FIG. 10 depicts an improved mobile imaging handset platform architecture.
- FIG. 11 depicts an improved mobile imaging service platform architecture.
- FIG. 12 depicts an OTN Upgrade of deployed MMSC Video Gateway.
- FIG. 13 depicts a Self-playing video MMS eliminates need for transcoding.
- FIG. 14 depicts a reduction in complexity, cost, and number of video editing servers re to deploy media producer services.
- FIG. 15 depicts a mobile video service platform.
- FIG. 16 depicts faster, lower cost development and deployment of higher quality multimedia services.
- FIG. 17 depicts aspects of mobile video services according to aspects of the present invention.
- FIG. 18 depicts Applications to Broadband Multimedia Devices and Services according to aspects of the present invention.
- FIG. 19 depicts Implementation Options for SW Imaging Application according to aspects of the present invention.
- FIG. 20 depicts Implementation Options for HW-Accelerated Imaging Application according to aspects of the present invention.
- FIG. 21 depicts Implementation Options for Hybrid HW-Accelerated SW Imaging Application according to aspects of the present invention.
- FIG. 22 depicts an Application: Simplified Multimedia Handset Platform Architecture.
- FIG. 23 depicts Elements of the mobile video messaging demo over a GSM/GPRS network.
- FIG. 24 depicts certain MMS functionality according to aspects of the present invention.
- a wavelet transform may comprise the repeated application of wavelet filter pairs to a set of data, either in one dimension or in more than one.
- a 2-D wavelet transform horizontal and vertical
- Video codecs according to the present invention can use a 3-D wavelet transform (horizontal, vertical, and temporal).
- An improved, symmetrical 3-D wavelet-based video compression/decompression (codec) device is desirable to reduce the computational complexity and power consumption in mobile devices well below those required for DCT-based codecs, as well as to enable simultaneous support for processing still images and video images in a single codec.
- Such simultaneous support for still images and video images in a single codec may eliminate or reduce the need for separate MPEG (video) and JPEG (still image) codecs, or greatly improve compression performance and hence storage efficiency with respect to Motion JPEG codecs.
- An improved, symmetrical 3-D wavelet-based video processing device is also desirable to reduce the computational complexity and power consumption in MMS infrastructure equipment utilized to support automated or manual editing of user-created video, as well as database storage, search, and retrieval of user-created video.
- aspects of the present invention comprise new methods, services and systems relating to innovative capture, compression, transmission, editing, storing and sharing video content associated with mobile devices.
- Aspects of the present invention may apply to telecom (both wireless and wireline providers) and internet, cable and other data and multimedia operators including fixed and mobile wireless service providers.
- Aspects of the present invention may provide for richer content, higher bandwidth usage and higher average revenue per user (ARPU).
- Mobile multimedia service (MMS) is the multimedia evolution of the text-based short message service (SMS). According to aspects of the present invention, a promising new MMS application presented is innovative video messaging and sharing, enabling a target audiences' need to communicate personal information.
- Mobile image messaging and sharing may require the addition of digital camera functionality (still images) and/or camcorder functionality (video images) to mobile handsets, so that subscribers can both capture (encoded) video messages that they wish to send, and play back (decode) video messages that they receive.
- digital camera functionality still images
- camcorder functionality video images
- Aspects of the present invention may also enable these functionalities in ways unavailable, if available at all, in the prior art.
- mobile devices may be enabled to evolve into integrated consumer multimedia entertainment platforms.
- a substantial investment in industry has been directed toward technologies and platforms that enable re-packaged broadcast television programming (such as news clips, sports highlights, and special “mobisodes” of popular TV programs) and other studio-generated video content (such as film previews and music videos) to be transmitted to and viewed on mobile devices.
- broadcast television programming such as news clips, sports highlights, and special “mobisodes” of popular TV programs
- other studio-generated video content such as film previews and music videos
- aspects of the present invention additionally enable significant reductions in the development cost and retail price of both camcorder phones and video messaging/sharing infrastructure equipment, which may be key to large scale commercial adoption of such devices and related mobile multimedia/data services, in both mature and emerging markets.
- Prior mobile image messaging/sharing services and applications are severely limited to capturing and transmitting much smaller-size and lower-frame-rate video images than those typically captured and displayed on other multimedia devices (see FIG. 1 ), such as TVs, personal computers, digital video camcorders, and personal media players.
- Mobile image messaging services and applications capable of supporting VGA (or larger) video at a frame rate of 30 fps or higher, as provided by aspects of the present invention, would be preferable.
- Video transmission over mobile networks is challenging in nature because of the higher data rates typically required, in comparison to the transmission of other data/media types such as text, audio, and still images.
- the limited and varying channel bandwidth, along with the fluctuating noise and error characteristics of mobile networks impose further constraints and difficulties on video transport.
- various joint source-channel coding techniques can be applied to adapt the video bit stream to different channel conditions (see FIG. 2 ).
- Such a joint source-channel coding approach according to aspects of the present invention may be scalable, in order to adapt to varying channel bandwidths and error characteristics.
- is supported scalability for multicast scenarios in which different devices at the receiving end of the video stream may have different limitations on decoding computational power and display capabilities.
- the source video sequence 30 may be first source coded 32 (i.e. compressed), followed by error correction code (ECC) channel coding 34 .
- ECC error correction code
- source coding typically uses such DCT-based compression techniques as, H.263, MPEG-4, or Motion JPEG.
- Example channel coding methods are Reed-Solomon codes, BCH codes, FEC codes, and turbo codes.
- the joint source and channel coded video bit stream then passes through the rate controller 36 to match the channel bandwidth requirement while achieving the best reconstructed video quality.
- the rate controller performs discrete rate-distortion computations on the compressed video bit stream before it sends the video bit stream for transmission over the channel 38 . Due to limitations in computational power in mobile devices, typical rate controllers only consider the available channel bandwidth, and do not explicitly consider the error characteristics of the transmission channel.
- a further benefit of such an improved adaptive joint-source channel coding technique is the corresponding ability of wireless carriers and MMS service providers to offer a greater range of quality-of-service (QoS) performance and pricing levels to their consumer and enterprise customers, thus maximizing the revenues generated using their wireless network infrastructure.
- QoS quality-of-service
- Multicast scenarios require a single adaptive video bit stream that can be decoded by many users. This is especially important in modern, large-scale, heterogeneous networks, in which network bandwidth limitations make it impractical to transmit multiple simulcast video signals specifically tuned for each user. Multicasting of a single adaptive video bit stream greatly reduces the bandwidth requirements, but requires generating a video bit stream that is decodable for multiple users, including high-end users with broadband wireless or wire line connections, and wireless phone users, with limited bandwidth and error-prone connections. Due to limitations in computational power in mobile devices, the granularity of adaptive rate controllers is typically very coarse, for example producing only a 2-layer bit stream including a base layer and one enhancement layer.
- imaging-enabled mobile handsets are limited to capturing smaller-size and lower-frame-rate video images than those typically captured and displayed on other multimedia devices, such as TVs, personal computers, digital video camcorders, and personal media players. These latter devices typically capture/display video images in VGA format (640 ⁇ 480 pixels) or larger, at a display rate of 30 frames-per-second (fps) or higher, whereas commercially available imaging-enabled mobile handsets are limited to capturing video images in QVGA format (320 ⁇ 240 pixels), QCIF format (176 ⁇ 144 pixels) or smaller, at a display rate of 15 fps or lower (See, e.g., FIG. 1 ). This reduced video capture capability is due to the excessive computational requirements, processor power consumption, and buffer memory required to complete the number, type, and sequence of computational steps associated with video compression/decompression using DCT transforms.
- Such handset architectures require codecs that utilize a combination of both software programs and hardware accelerators running on a combination of reduced instructions set (RISC) processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), and reconfigurable processing devices (RPDs), together with larger buffer memory blocks (typical memory capacity of 1 Mbyte or more).
- RISC reduced instructions set
- DSPs digital signal processors
- ASICs application-specific integrated circuits
- RPDs reconfigurable processing devices
- codec functions may be implemented using such RISC processors, DSPs, ASICs, multimedia processors, and RPDs as separate integrated circuits (ICs), or may combine one or more of the RISC processors, DSPs, ASICs, multimedia processors, and RPDs integrated together in a system-in-a-package (SIP) or system-on-a-chip (SoC).
- SIP system-in-a-package
- SoC system-on-a-chip
- Codec functions running on RISC processors or DSPs are typically software routines, with the advantage that they can be modified in order to correct programming errors or upgrade functionality.
- the disadvantage of implementing certain complex, repetitive codec functions as software is that the resulting overall processor resource and power consumption requirements typically exceeds those available in mobile communications devices.
- Codec functions running on ASICs and multimedia processors are typically fixed hardware implementations of complex, repetitive computational steps, with, typically, the advantage that specially tailored hardware acceleration can substantially reduce the overall power consumption of the codec.
- the disadvantages of implementing certain codec functions in fixed hardware include longer and more expensive design cycles, the risk of expensive product recalls in the case where errors are found in the fixed silicon implementation, and the inability to upgrade fixed silicon functions in deployed products in the case where newly developed features are to be added to the imaging application.
- Codec functions running on RPDs are typically routines that require both hardware acceleration and the ability to add or modify functionality in final mobile imaging handset products.
- the disadvantage of implementing certain codec functions on RPDs is the larger number of silicon gates and higher power consumption required to support hardware reconfigurability in comparison to fixed ASIC implementations.
- An imaging application that reduces or eliminates complex, repetitive codec functions so as to enable mobile imaging handsets capable of capturing VGA (or larger) video at a frame rate of 30 fps with an all-software architecture would be preferable, in order to simplify the above architecture and enable handset costs compatible with high-volume commercial deployment.
- the present invention is the first technology to successfully accomplish and enable these objectives.
- Multimedia handsets are required not only to support picture and video messaging capabilities, but also a variety of additional multimedia capabilities (voice, music, graphics) and a variety of fixed and mobile wireless access modes, including but not limited to 2.5 G and 3 G cellular access, WiBro, HSDPA, WiFi, wireless LAN, and Bluetooth.
- OTA over-the-air
- An all-SW imaging application would be preferable to enable OTA distribution and management of the imaging application by handset manufacturers, mobile operators, and other MMS service providers. Again, the present invention is the first technology to successfully enable these objectives.
- Java technology brings a wide range of devices, from servers to desktops to mobile devices, together under one language and one technology. While the applications for this range of devices differ, Java technology works to bridge those differences where it counts, allowing developers who are functional in one area to leverage their skills across a wide spectrum of devices and applications.
- J2ME Java 2, Micro Edition
- Sun redefined the architecture of the Java technology, grouping it into three editions.
- Standard Edition (J2SE) offered a practical solution for desktop development and low-end business applications.
- Enterprise Edition (J2EE) was for developers specializing in applications for the enterprise environment.
- Micro Edition (J2ME) was introduced for developers working on devices with limited hardware resources, such as PDAs, cell phones, pagers, television set top boxes, remote telemetry units, and many other consumer electronic and embedded devices.
- J2ME is aimed at machines with as little as 128 KB of RAM and with processors a lot less powerful than those used on typical desktop and server computers.
- J2ME actually consists of a set of profiles. Each profile is defined for a particular type of device—cell phones, PDAs, etc.—and consists of a minimum set of class libraries required for the particular type of device and a specification of a Java virtual machine required to support the device.
- the virtual machine specified in any J2ME profile is not necessarily the same as the virtual machine used in Java 2 Standard Edition (J2SE) and Java 2 Enterprise Edition (J2EE).
- Sun identified within each of these two categories classes of devices with similar roles—so, for example, all cell phones fell within one class, regardless of manufacturer. With the help of its partners in the Java Community Process (JCP), Sun then defined additional functionality specific to each class of devices.
- JCP Java Community Process
- a configuration may be a Java virtual machine (JVM) and a minimal set of class libraries and APIs providing a run-time environment for a select group of devices.
- JVM Java virtual machine
- a configuration may specify a least common denominator subset of the Java language, one that fits within the resource constraints imposed by the family of devices for which it was developed. Because there is such great variability across user interface, function, and usage, a typical configuration does not define such important pieces as the user interface toolkit and persistent storage APIs. The definition of that functionality belongs, instead, to what is called a profile.
- a J2ME profile may be a set of Java APIs specified by an industry-led group that is meant to address a specific class of device, such as pagers and cell phones. Each profile is built on top of the least common denominator subset of the Java language provided by its configuration, and is meant to supplement that configuration.
- Two profiles important to mobile handheld devices are: the Foundation profile, which supplements the CDC, and the Mobile Information Device Profile (MIDP), which supplements the CLDC. More profiles are in the works, and specifications and reference implementations continue to be developed and released.
- JSR 185 The Java Technology for the Wireless Industry (JTWI) specification, JSR 185, defines the industry-standard platform for the next generation of Java technology-enabled mobile phones.
- JTWI is defined through the Java Community Process (JCP) by an expert group of leading mobile device manufacturers, wireless carriers, and software vendors.
- JTWI specifies the technologies that must be included in all JTWI-compliant devices: CLDC 1.0 (JSR 30), MIDP 2.0 (JSR 118), and WMA 1.1 (JSR 120), as well as CLDC 1.1 (JRS 139) and MMAPI (JSR 135) where applicable.
- JSR-135 Mobile Media API
- JSR-234 Advanced Multimedia Supplements
- JTWI JTWI specification raises the bar of functionality for high-volume devices, while minimizing API fragmentation and broadening the substantial base of applications that have already been developed for mobile phones.
- Benefits of JTWI include:
- a Java-based imaging application would be preferable for “write-once, run-anywhere” portability across all Java-enabled handsets, for Java VM security and handset/network robustness against viruses, worms, and other mobile network security “attacks”, and for simplified OTA codec and application download procedures.
- Such a Java-based imaging application should conform to JTWI specifications JSR-135 (“Mobile Media API”) and JSR-234 (“Advanced Multimedia Supplements”). Aspects of the present invention provide these advantages.
- Key components of a mobile imaging service platform architecture may include:
- Typical functions included in the MMSC according to aspects of the present invention include:
- the video gateway in an MMSC may serve to transcode between the different video formats that are supported by the imaging service platform. Transcoding is also utilized by wireless operators to support different voice codecs used in mobile telephone networks, and the corresponding voice transcoders are integrated into the RNC. Upgrading such a mobile imaging service platform with the architecture shown in FIG. 4 if found in prior architectures, would typically require deploying new handsets, and manually adding new hardware to the MMSC video gateway. In some mobile video messaging and sharing applications, it may be desirable to eliminate the cost and complexity associated with transcoding.
- One aspect of the current invention is the ability to embed a software decoder with each transmitted video stream, enabling “self-playing” functionality on common handset and PV video players.
- the MMS applications servers in an MMSC may support applications such as automated or manual editing of user-created video, as well as database storage, search, and retrieval of user-created video.
- the computational complexity required to implement such functions requires specialized servers to be installed by mobile operators, with the corresponding video processing functions typically requiring expensive and high-power application-specific integrated circuits (ASICs) and digital signal processors (DSPs), rather than simpler SW applications running on less-expensive and lower-power CPU chips used in standard personal computers (PCs) and servers.
- ASICs application-specific integrated circuits
- DSPs digital signal processors
- an all-software mobile imaging applications service platform would be preferable, in order to support automated OTA upgrade of deployed handsets, automated OTN upgrade of deployed MMSCs, and support for mobile video applications using standard PCs and servers.
- a Java implementation of the mobile handset imaging application may be preferable in terms of improved handset/network robustness against viruses, worms, and other “attacks”, allowing mobile network operators to provide the quality and reliability of service required by national regulators.
- Upgrading MMSC infrastructure is also costly if new or specialized hardware is required.
- An all-SW applications and service platform would be preferable in order to enable automated OTA upgrade of handsets, OTN upgrade of MMSC video gateways, and support for mobile video applications using standard PCs and servers.
- the need for transcoding between different video formats also adds additional cost and complexity.
- an all-SW video codec solution substantially reduces or eliminates baseband processor and video accelerator costs and requirements in multimedia handsets. Combined with the ability to install the codec post-production via OTA download, this all-SW solution substantially reduces the complexity, risk, and cost of both handset development and video messaging service architecture and deployment.
- SW video transcoders and editing, storing, searching, retrieval applications enable automated over-the-network (OTN) upgrade of deployed MMS control (MMSC) infrastructure, as well as the use of standard PCs and servers to run such applications.
- OTN over-the-network
- MMSC deployed MMS control
- the present invention wavelet transcoders provide carriers with complete interoperability between the wavelet video format and other standards-based and proprietary video formats.
- the present invention also allows a software decoder to be embedded with each transmitted video stream, enabling “self-playing” functionality on common handset and PV video players, and eliminating the cost and complexity of transcoding altogether.
- the present invention's all-SW video platform allows rapid deployment of new MMS services, also parts of embodiments of the present invention, that leverage processing speed and video production accuracy not available with other existing technologies.
- the present invention's wavelet codecs are also unique in their ability to efficiently process both still images and video, and can thus replace separate MPEG and JPEG codecs with a single lower-cost and lower-power solution that can simultaneously support both mobile picture-mail and video-messaging services.
- FIG. 5 provides a comparison of the relative computational requirements of a traditional DCT encoder technology and exemplary technologies of the present invention.
- the application of a wavelet transform stage also enables design of quantization and entropy-coding stages with greatly reduced computational complexity.
- Further advantages of the 3-D wavelet codecs of the present invention for mobile imaging applications, devices, and services include:
- wavelet transforms using short dyadic integer filter coefficients in the lifting structure in certain embodiments the Haar, 2-6, and 5-3 wavelets and variations of them are used. These can require only adds, subtracts, and small fixed shifts—no multiplication or floating-point operations are needed.
- Lifting Scheme computation These filters can be computed using the Lifting Scheme which allows in-place computation. This minimizes use of registers and temporary RAM locations, and keeps references local for highly efficient use of caches.
- Wavelet transforms in pyramid form with customized pyramid structure Certain embodiments of the present invention compute each level of the wavelet transform sequence on half of the data resulting from the previous wavelet level, so that the total computation is almost independent of the number of levels. Aspects of the present invention customize the pyramid to leverage the advantages of the Lifting Scheme above and further economize on register usage and cache memory bandwidth.
- Block structure In contrast to most wavelet compression implementations, aspects of the present invention may divide the picture into rectangular blocks and processes each block separately from the others. This allows memory references to be kept local and to do an entire transform pyramid with data that remains in the processor cache, saving a significant amount of data movement within most processors.
- the present block structure is particularly helpful in HW embodiments as it avoids the requirement for large intermediate storage capacity in the signal flow.
- Block boundary filters the present invention may also use modified filter computations at the boundaries of each block that avoid sharp artifacts as set out in U.S. patent application Ser. No. 10/418,363, incorporated herein by reference.
- Chroma temporal removal aspects of the present invention may also avoid processing the chroma-difference signals for every field, instead using a single field of chroma for a GOP as set out in U.S. patent application Ser. No. 10/447,514, incorporated herein by reference.
- Temporal compression using 3D wavelets Certain embodiments of the present invention may not use the very expensive motion-search and motion-compensation operations of conventional video compression methods such as MPEG. Instead those embodiments compute a field-to-field temporal wavelet transform. This is much less expensive to compute. Also sometimes used are short integer filters with the Lifting Scheme in this aspect.
- the quantization step of the compression process may be accomplished using a binary shift operation uniformly over a range of coefficient locations. This avoids the per-sample multiplication or division required by conventional quantization.
- Piling In certain embodiments of the present invention, the amount of data to be handled by the following entropy coder by doing run-of-zeros conversion is reduced. In certain embodiments, the methods and inventions disclosed in U.S. patent application Ser. No. 10/447,455 are utilized for a counting runs of zeros on parallel processing architectures.
- Cycle-efficient entropy coding In certain embodiments of the present invention, the entropy coding step of the compression process is accomplished using techniques that combine the traditional table lookup with direct computation on the input symbol. Because the symbol distribution has been characterized, such simple entropy coders as Rice-Golomb or exp-Golomb or Dyadic Monotonic can be used. The choice of entropy coder details will often vary depending on the processor platform capabilities.
- the fine grain scalability of the wavelet-based codec enables improved adaptive rate control, multicasting, and joint source-channel coding.
- the reduced computational complexity and higher computational efficiency of the present wavelet algorithms allows information on both instantaneous and predicted channel bandwidth and error conditions to be utilized in all three of the source coder, the channel coder, and the rate controller to maximize control of both the instantaneous and average quality (video rate vs. distortion) of the reconstructed video signal (see FIG. 6 ).
- the improved adaptive joint-source channel coding technique of the present invention allows wireless carriers and MMS service providers to offer a greater range of quality-of-service (QoS) performance and pricing levels to their consumer and enterprise customers. Utilizing improved adaptive joint-source channel coding based on algorithms with higher computational efficiency enables support for a much higher level of network heterogeneity, in terms of channel types (wireless and wire line), channel bandwidths, channel noise/error characteristics, user devices, and user services.
- FIG. 7 illustrates an improved mobile imaging handset platform architecture according to aspects and embodiments of the present invention.
- the imaging application according to aspects of the present invention is implemented as an all-software application running as native code or as a Java application on a RISC processor. Acceleration of the Java code operation may be implemented within the RISC processor itself, or using a separate Java accelerator IC. Such a Java accelerator may be implemented as a stand-alone IC, or this IC may be integrated with other functions in either a SIP or SoC.
- the improved mobile imaging handset platform architecture illustrated in FIG. 7 eliminates the need for separate DSP, ASIC, multimedia processor, or RFD processing blocks for the mobile imaging application, (as would be required in prior devices or systems) and also greatly reduces the buffer memory requirements for image processing in the mobile handset.
- FIG. 8 shows the reduction in computational requirements for full VGA 30 fps video encoding provided by aspects of the current invention, in comparison to current state-of-the-art industry solutions reached after the filing date of the present application's priority filing based upon MPEG-4 and H-264 video codecs.
- FIG. 9 shows one implementation of aspects of the current invention on a commercial mobile GSM camcorder phone platform.
- the existing GSM baseband/multimedia SoC (Texas Instruments OMAP 850 shown in FIG. 9 ) requires a HW accelerator, a DSP, and a RISC processor for QCIF/15 fps camcorder functionality
- the present invention provides VGA/30 fps camcorder functionality on this platform using only SW running on the RISC processor without the need of a HW accelerator or a DSP.
- FIG. 10 shows one implementation of aspects of the current invention on a commercial mobile CDMA camcorder phone platform.
- the existing CDMA baseband/multimedia SoC (Qualcomm MSM6500 shown in FIG. 10 ) requires a HW accelerator, a DSP, and a RISC processor for QCIF/15 fps camcorder functionality
- the present invention provides VGA/30 fps camcorder functionality on this platform using only SW running on the RISC processor without the HW accelerator or DSP.
- Components of an improved mobile imaging service platform architecture may include:
- Typical functions included in the MMSC may include:
- certain steps involved in deploying the improved imaging service platform may include:
- Video Gateway Transcoder application and/or video messaging/sharing applications are available for updating deployed MMSCs.
- the update can be installed via automated OTN deployment or via manual procedures.
- FIG. 13 shows “self-playing” video MMS functionality achieved by integrating the SW decoder with the transmitted video stream.
- FIG. 14 shows the reduction in complexity, cost, and number of video application servers required to deploy media producer services such as automated or manual editing of user-created video, as well as database storage, search, and retrieval of user-created video.
- FIG. 15 shows the functional elements of a video messaging/sharing/calling platform incorporating the improved wavelet-based codec/camcorder application, improved joint source channel coding, and improved video editing and database storage, search, and retrieval.
- FIG. 16 shows the benefits in terms of faster, lower cost development and deployment of higher quality multimedia handsets & services, including the ability to deploy an innovative personal multi-media market place platform in which users can preview, share, buy, and sell “soft” copies (download) or “hard” copies (DVD) of user-created audio/video content.
- the present invention also allows for more efficient video “tagging” for database indexing and network (RSS) feeds, and supports interfaces to existing web-based market places such as E-bay, Google, Yahoo, Microsoft, and other portals.
- RSS database indexing and network
- FIG. 17 shows several innovative new mobile video services based on the improved wavelet-based codec/camcorder application, improved joint source channel coding, and improved video editing and database storage, search, and retrieval.
- FIG. 18 shows applications of the above video messaging/sharing/calling platform incorporating the improved wavelet-based codec/camcorder application, improved joint source channel coding, and improved video editing and database storage, search, and retrieval, to deploy new video services on fixed wireless, mobile wireless, and wireline networks, as well as “converged” networks combining elements of fixed wireless, mobile wireless, and wireline architectures.
- aspects of the present invention with their improved wavelet-based mobile video imaging application, joint source-channel coding, handset architecture, and service platform architecture achieve goals of higher mobile video image quality, lower handset cost and complexity, and reduced service deployment costs.
- Various embodiments of aspects of the present invention provide enhancements to the mobile imaging handset architecture.
- the imaging application can be installed via OTA download ( 400 a , 400 b , 400 c ) to the baseband multimedia processing section of the handset 402 a , to a removable storage device 402 b , or to the imaging module 402 c .
- the imaging application can also be installed during manufacturing or at point-of-sale to the baseband multimedia processing section of the handset, to a removable storage device, or to the imaging module. Additional implementation options are also possible as mobile device architectures evolve.
- performance of the mobile imaging handset may be further improved, and costs and power consumption may be further reduced, by accelerating some computational elements via HW-based processing resources in order to take advantage of ongoing advances in mobile device computational HW (ASIC, DSP, multimedia processor, RPD) and integration technologies (SoC, SIP).
- ASIC mobile device computational HW
- DSP digital signal processor
- RPD multimedia processor
- SoC integration technologies
- hybrid architectures offered by aspects of the present invention for the imaging application may offer enhancements by implementing some computationally intensive, repetitive, fixed functions in HW, and implementing in SW those functions for which post-manufacturing modification may be desirable or required.
- FIG. 22 shows potential simplifications in mobile camcorder device architecture, deployment, and maintenance.
- the all-SW imaging solution of the present invention substantially reduces baseband processor and video accelerator costs and requirements in multimedia handsets. Combined with the ability to install and maintain the codec post-production via OTA download, this all-SW solution can substantially reduce the complexity, risk, and cost of both handset development and video messaging service deployment.
- the present invention provides mobile operators with the first mobile video messaging and sharing platform that delivers the video quality, mobile handset price-point, and service deployment costs required for mass-market adoption by consumer and enterprise customers.
- the present invention provides the first all-SW camcorder phone application capable of real-time capture of full (VGA)-size images (640 ⁇ 480 pixels) at 30 frames per second (fps), using only according to certain aspects and embodiments of the present invention standard RISC processors already incorporated in the vast majority of multimedia handsets.
- VGA full
- fps frames per second
- the present invention's low-complexity video processing and distribution technologies can be integrated into a powerful new all-software platform that enables turnkey deployment using existing mobile handsets and mobile Multimedia Messaging Service (MMS) infrastructure.
- MMS mobile Multimedia Messaging Service
- aspects of the present invention's content management platform provide carriers with modules for integrating compressed images and videos, according to the present technology, together with sounds and text into complete mobile multimedia messages and “ring-tones”, along with on-the-fly editing, thumbnail previews, multimedia mailboxes, on-line repository, sharing, and marketing services, and subscription management.
- Example 1 describes the components, setup, and operation of an introductory demonstration of the functionality and benefits provided by an embodiment of aspects of the present invention's software-only mobile video messaging platform.
- the demo utilizes commercially available GSM/GPRS multimedia handsets, and was designed to operate over any commercial GSM/GPRS network. The demonstration operated very successfully.
- the demo can also be readily adapted to utilize CDMA handsets, and to operate over any commercial CDMA network.
- the demo in Example 1 runs a demo and set of files code named “Droplet” and so labeled in Example 1.
- the demo includes the following five elements:
- a UNIX-based remote server is used to perform the following MMS functions:
- the compressed DV video files captured by the camcorder are first converted into decompressed UYVY video format in the PC, and then input to the MDA-II handset for encoding/compression by the present invention's DTV codec.
- UYVY is a typical video format that would be input to the video codec in a multimedia handset.
- Virtual Dub is used to convert the compressed DV video files, as captured by the camcorder, into decompressed UYVY video format in the PC. These decompressed video files are then input to the MDA-II handset for encoding/compression by the present invention's DTV codec.
- UYVY is a typical video format that would be input to the video codec in a multimedia handset.
- the remote MMS server functions both as an FTP server (to enable download of video codec files to the handsets, and network storage of video files from the recording handset), and as a mail server (to enable email/SMS notification and download of video messages by networked computers and other handsets). Functionally, the server must be able to send SMS messages, in order to enable SMS notification to other handsets of pending video messages.
- the timeout period is increased to greater than the default setting of 60 sec.
- the handset manufacturer HTC has provided the recommended registry changes. If there is no Registry Editor installed on the device, first install the registry editor included in Droplet's Demo package under the “PHMRegEditor” directory.
- source code for the MDA-II Handset UI Application is available under the directory MDA_DTV ⁇ mms_client_src as a reference.
- the MDA-II has now been configured for the demo.
- This step is optional, since the default video player on the MDA-II can view the decoded Droplet video file.
- the timeout period needs to be increased to greater than the default setting of 60 sec.
- the handset manufacturer HTC has provided the recommended registry changes. If there is no Registry Editor installed on the device, first install the registry editor included in Droplet's Demo package under the “PHMRegEditor” directory.
- the MDA-II handset is used to encode/compress high quality uncompressed live video input from an external video camcorder. While the MDA-II has a VGA capture camera, it can only capture still images at that resolution. The video capture of the MDA-II is limited to QCIF, 10 fps video that is automatically compressed in 3GPP format on the device.
- This section will demonstrate the ability to send the compressed QCIF/15 fps video from the MDA-II handset to the MMS server via GPRS. From there, an SMS notification will be sent to the targeted handset (in this case the Xphone), indicating that a video MMS is ready for download and playback. Alternatively, an email notification will be sent if the targeted receiving device is a networked computer.
- the targeted handset in this case the Xphone
- This section will demonstrate the ability to receive the SMS notification on the Xphone, and to connect to the MMS server and download the QCIF/15 fps video file together with the DTV decoder. Upon receipt of the video file and decoder, the file will be decoded and played on the Xphone.
- aspects of the present invention comprise, in part, an all-software camcorder phone application capable of real-time capture of full (VGA)-size images (640 ⁇ 480) at 30 frames per second (fps), which may use only a single standard RISC processor already incorporated in the vast majority of multimedia handsets.
- VGA full
- fps frames per second
- current MPEG-based camcorder phones support real-time capture of images that are limited to QCIF or CIF size ( 1/16th or 1 ⁇ 4 the size of VGA) at 4-15 fps.
- QCIF or CIF size 1/16th or 1 ⁇ 4 the size of VGA
- choppy video clips require complex and expensive handset platform designs, in which the video functions are implemented as a combination of hardware and software, and partitioned between multiple processing devices: RISC processors, ASIC, and DSPs.
- aspects of the present invention's low-complexity video processing and distribution technologies are integrated into a powerful new and inventive all-software video messaging platform that enables turnkey deployment using existing mobile handsets and mobile Multimedia Messaging Service controller (MMSC) infrastructure.
- MMSC mobile Multimedia Messaging Service controller
- embodiments of the present invention's content management platform provide modules for integrating the invention's compressed images and videos together with sounds and text into complete mobile multimedia messages and “ring-tones”, along with on-the-fly editing, thumbnail previews, multimedia mailboxes, on-line repository services, and subscription management.
- aspects of the present invention's video codecs offer customers a 30-40 ⁇ reduction in power consumption (both SW and HW implementations—see Table 1) when compared to optimized MPEG-2/MPEG-4 codecs.
- HW product implementation costs are significantly reduced via a 10 ⁇ reduction in the number of CMOS gates required, from approximate ⁇ 1 million to ⁇ 100,000, and hence in the corresponding silicon real estate requirements.
- VGA full size
- 30 fps full-frame-rate
- the present invention's innovative video codec designs also reduce internal memory requirements from several megabytes to 128 kilobytes, freeing up on-board memory resources in mobile handsets for other revenue-generating features and applications.
- the present invention's codecs are also able to efficiently process both still images and video, and can thus replace separate MPEG and JPEG codecs with a single lower-cost and lower-power solution.
- the present invention's unique mobile video platform technologies also offer significant benefits across a broad range of other mobile video services, via a combination of: scalable image size: QCIF (176 ⁇ 144)-D1 (720 ⁇ 480), simplified video editing (cuts, inserts, text overlays, etc.), simplified synchronization with voice codecs, and low latency for enhanced video streaming performance.
- scalable image size QCIF (176 ⁇ 144)-D1 (720 ⁇ 480)
- simplified video editing cuts, inserts, text overlays, etc.
- simplified synchronization with voice codecs and low latency for enhanced video streaming performance.
- the present invention also comprises MMS infrastructure products enabling deployment of premium video messaging services in conjunction with the inventive SW video codec/camcorder applications for mobile handsets. Additional aspects of the invention comprise advanced transcoding applications support complete interoperability with other commonly-deployed standards-based and proprietary video formats. Additionally included is a content management platform that provides modules for integrating the invention's compressed images and videos together with sounds and text into complete mobile multimedia messages and “ring-tones”, along with a suite of corresponding MMS message management capabilities. This content management platform can be used by wireless operators and MMS service providers both as a set of SW modules, for rapid and cost-effective upgrades to existing MMS infrastructure, and as a stand-alone server for new MMS controller installations.
- the inventive MMS infrastructure products may include:
- the present invention also comprises a Content Management Service Platform that with SW Modules or Stand-Alone Server may include:
- JSR-135 Mobile Media API Specification
- the present invention's DTV-JVC Java Video Codec generates decompressed video images that support all Player Functionality defined in Java Community Process JSR-135 including the following: Int getDisplayHeight( ) Return the actual height of the current render video. Int getDisplayWidth( ) Return the actual width of the current render video. Int getDisplayX( ) Return the X-coordinate of the video with respect to the GUI object where the video is displayed. Int getDisplayY( ) Return the Y-coordinate of the video with respective to the GUI object where the video is displayed. byte[ ] getSnapshot(java.lang.String imageType) Get a snapshot of the displayed content. Int getSourceHeight( ) Return the height of the source video.
- java.lang.Object initDisplayMode (int mode, java.lang.Object arg) Initialize the mode on how the video is displayed.
- Void setDisplayFullScreen (boolean fullScreenMode) Set the size of the render region for the video clip to be fullscreen.
- Void setDisplayLocation (int x, int y) Set the location of the video with respect to the canvas where the video is displayed.
- Void setVisible (boolean visible) Show or hide the video.
- JSR-234 Advanced Multimedia Supplements
- the present invention's DTV-JVC Java Video Codec generates decompressed video images that support all Player Effect Controls defined in Java Community Process JSR-234 including the following:
- ImageFilterControl is an image effect that can be used to set various image filters such as monochrome and negative.
- ImageTonalityControl is an effect that can be used to set various image settings such as brightness, contrast, and gamma.
- ImageTransformControl is used to crop, zoom, mirror, flip, stretch, and rotate images.
- OverlayControl controls the setting of overlay images on top of video or still images.
- WhiteBalanceControl is an image/video effect for altering the white balance.
- the present invention also comprises products, methods and processes for establishing, providing and operating a mobile video blog service.
- This service provides every user having a video phone with the ability to: shoot, edit, save, share, and “publish” their personal videos and movies online.
- Improved adaptive joint-source channel coding technique is the corresponding ability of wireless carriers and MMS service providers to offer a greater range of quality-of-service (QoS) performance and pricing levels to their consumer and enterprise customers, thus maximizing the revenues generated using their wireless network infrastructure.
- QoS quality-of-service
- Improved adaptive joint-source channel coding based on algorithms with higher computational efficiency, enables support for a much higher level of network homogeneity, in terms of channel types (wireless and wire line), channel bandwidths, channel noise/error characteristics, user devices, and user services.
- methods, devices, processes and business methods providing innovative and enhanced services in the field of still and moving video in the mobile telephone fields.
- QoS quality-of-service
- imaging-enabled mobile handset architecture using as pects and features of the preceding paragraphs of this summary, where the mobile imaging application is incorporated in the handset baseband multimedia processing section of the handset, in the imager module, or in a removable storage medium.
- a mobile imaging transcoder for universal compatibility of the above features of this summary with other standards-based or proprietary imaging formats—all SW application delivered to and installed in an MMSC Video Gateway via automated OTN upgrade or via manual procedures.
Abstract
Systems and methods are provided for compressing and decompressing still image and video image data in mobile devices and monitoring applications. Corresponding mobile device and camera architectures, and service platform architectures for transmitting, storing, editing and transcoding still images and video images over wireless and wired networks and viewing them on display-enabled devices are also provided.
Description
- The present application claims priority to U.S. provisional application No. 60/654,058 filed Feb. 16, 2005.
- The present application also incorporates by reference in its entirety U.S. application Ser. No. 11/232,165 filed Sep. 20, 2005; U.S. application Ser. No. 11/232,726 filed Sep. 21, 2005; U.S. application Ser. No. 11/232,725 filed Sep. 21, 2005; U.S. application Ser. No. 11/249,561 filed Oct. 12, 2005; and U.S. application Ser. No. 11/250,797 filed Oct. 13, 2005.
- This application incorporates the following in their entirety for all purposes: Sweldens, Wim, The Lifting Scheme: A custom-design construction of biorthogonal wavelets. Appl. Comput. Harmon. Anal. 3(2):186-200, 1996; U.S. patent application Ser. No. 10/418,363 Filed: Apr. 17, 2003 entitled WAVELET TRANSFORM SYSTEM, METHOD AND COMPUTER PROGRAM PRODUCT with named inventors William C. Lynch, Krasimir D. Kolarov, and Steven E. Saunders; U.S. patent application Ser. No. 10/447,514 Filed: May 28, 2003 CHROMA TEMPORAL RATE REDUCTION AND HIGH-QUALITY PAUSE SYSTEM AND METHOD with named inventors Steven E. Saunders, Krasimir D. Kolarov, and William C. Lynch; U.S. patent application Ser. No. 10/447,455 Filed: May 28, 2003 PILE PROCESSING SYSTEM AND METHOD FOR PARALLEL PROCESSOR with named inventors William C. Lynch, Krasimir D. Kolarov, and Steven E. Saunders; Golomb, S. W. (1966). “Run-length encodings.” IEEE Transactions on Information Theory, IT-12(3):399-401; R. F. Rice, “Some Practical Universal Noiseless Coding Techniques,” Jet Propulsion Laboratory, Pasadena, Calif., JPL Publication 79-22, March 1979; J. Teuhola, “A Compression Method for Clustered Bit-Vectors,” Information Processing Letters, vol. 7, pp. 308-311, October 1978; U.S. patent application Ser. No. 10/447,455 Filed: May 28, 2003
- PILE PROCESSING SYSTEM AND METHOD FOR PARALLEL PROCESSORS with named inventors William C. Lynch, Krasimir D. Kolarov, and Steven E. Saunders.
- The present application is a continuation-in-part of U.S. patent application Ser. No. 11/232,726 filed Sep. 21, 2005 entitled “Multiple Technique Entropy Coding System and Method” which claims priority from U.S. Provisional Application No. 60/612,652 filed Sep. 22, 2004; continuation-in-part of U.S. patent application Ser. No. 11/232,725 filed Sep. 21, 2005 entitled “Permutation Procrastination” which claims priority from U.S. Provisional Application No. 60/612,651 filed Sep. 22, 2004; continuation-in-part of U.S. application Ser. No. 11/232,165 filed Sep. 20, 2005 entitled “Compression Rate Control System and Method with Variable Subband Processing” which claims priority from U.S. Provisional Application No. 60/612,311 filed Sep. 21, 2004; continuation-in-part of U.S. patent application Ser. No. 10/955,240 filed Sep. 29, 2004 entitled “System and Method for Temporal Out-of-Order Compression and Multi-Source Compression Rate Control” now U.S. Publication No. US 2005/0105609 published on May 19, 2005, which claims priority from U.S. Provisional Application No. 60/612,311 filed Sep. 22, 2004, U.S. Provisional Application No. 60/507,148 and U.S. Provisional Application No. 60/507,147 both filed Sep. 30, 2003; continuation-in-part of U.S. patent application Ser. No. 10/944,437 filed Sep. 16, 2004 entitled “Multiple Codec-Imager System and Method” now U.S. Publication No. US 2005/0104752 published on May 19, 2005, which is a continuation of U.S. Pat. No. 6,825,780 issued Nov. 30, 2004 which claims priority from U.S. Provisional Application No. 60/390,380 filed Jun. 21, 2002 and U.S. Provisional Application No. 60/374,061 filed Apr. 19, 2002; continuation-in-part of U.S. patent application Ser. No. 10/447,455 filed on May 28, 2003 entitled “Pile-Processing System and Method for Parallel Processors” now U.S. Publication No. US 2003/0229773 published on Dec. 11, 2003, which claims priority from U.S. Provisional Application Nos. 60/385,253 and 60/385,250 both filed on May 28, 2002; continuation-in-part of U.S. patent application Ser. No. 10/447,514 filed on May 28, 2003 entitled “Chroma Temporal Rate Reduction and High-Quality Pause System and Method” now U.S. Publication No. US 2003/0235340 published on Dec. 25, 2003; which claims priority from U.S. Provisional Application Nos. 60/390,345 and 60/390,492 both filed on Jun. 21, 2002; continuation-in-part of U.S. patent application Ser. No. 10/418,649 filed Apr. 17, 2003 entitled “System, Method and Computer Program Product for Image and Video Transcoding” now U.S. Publication No. US 2003/0206597 published on Nov. 6, 2003, which claims priority from U.S. Provisional Application No. 60/374,069 filed Apr. 19, 2002; continuation-in-part of U.S. patent application Ser. No. 10/418,363 filed Apr. 17, 2003 entitled “Wavelet Transform System, Method and Computer Program Product” now U.S. Publication No. US 2003/0198395 published on Oct. 23, 2003, which claims priority from U.S. Provisional Patent Application No. 60/390,383 filed on Jun. 21, 2002, U.S. Provisional Patent Application No. 60/385,254 filed May 28, 2002 and U.S. Provisional Application Nos. 60/373,974 and 60/373,966 both filed on Apr. 19, 2002; each of which is incorporated herein by reference in its entirety.
- This application also incorporates by reference in its entirety U.S. Pat. No. 6,847,317 issued on Jan. 25, 2005 entitled “System and Method for a Dyadic-Monotonic (DM) Codec”; and U.S. Pat. No. 6,825,780 issued on Nov. 30, 2004 entitled “Multiple Codec-Imager System and Method.”
- Directly digitized images and video take lots of bits; it is common to compress images and video for storage, transmission, and other uses. Several basic methods of compression are known, and very many specific variants of these. A general method can be characterized by a three-stage process: transform, quantize, and entropy-code. Most image and video compressors share this basic architecture, with variations.
- The intent of the transform stage in a video compressor is to gather the energy or information of the source picture into as compact a form as possible by taking advantage of local similarities and patterns in the picture or sequence. No compressor can possibly compress all possible inputs; we design compressors to work well on “typical” inputs and ignore their failure to compress “random” or “pathological” inputs. Many image compression and video compression methods, such as MPEG-2 and MPEG-4, use the discrete cosine transform (DCT) as the transform stage. Some newer image compression and video compression methods, such as MPEG-4 static texture compression, use various wavelet transforms as the transform stage.
- Quantization discards information after the transform stage; the reconstructed decompressed image cannot then be an exact reproduction of the original. Entropy coding is generally a lossless step: this step takes the information remaining after quantization and codes it so that it can be reproduced exactly in the decoder. Thus the design decisions about what information to discard is not affected by the following entropy-coding stage.
- A limitation of DCT-based video compression/decompression (codec) techniques is that, having been developed originally for video broadcast and streaming applications, they rely on the encoding of video content in a studio environment, where high-complexity encoders can be run on computer workstations. Such computationally complex encoders allow computationally simple and relatively inexpensive decoders (players) to be installed in consumer playback devices. However, such asymmetric encode/decode technologies are a poor match to mobile multimedia devices, in which video messages must be captured in real time in the handset itself, as well as played back. As a result, video in mobile devices is typically limited to much smaller sizes and much lower frame rates than in other consumer products.
- This invention pertains to methods, devices, systems, and architectures relating to still image and video image recording in devices, including mobile devices, corresponding mobile device architectures, service platform architectures and methods and services for transmitting, storing, editing, sharing, marketing, and transcoding still images and video images over wireless and wired networks and systems and viewing them on display-enabled devices, as well as network and other system services in relation to the foregoing. The present invention also pertains to improvements in the image recording technique, and corresponding improvements in the architectures of mobile devices and service platforms.
- Aspects of the present invention comprise all-software video codecs/camcorder applications for compressing and/or decompressing video or still images. Aspects of the present invention also comprise infrastructure products, methods and processes, including mobile multimedia service (MMS) infrastructure applications, for deploying video messaging and sharing services in conjunction with software video codec/camcorder applications for mobile handsets as well as editing and transcoding applications to support complete interoperability with other commonly-deployed standards-based and proprietary video formats. Aspects of the invention also comprise methods, processes and business processes for establishing, enabling, distributing and operating innovative MMS services, including an innovative mobile video blog and marketing service for video content created by mobile users on mobile devices.
-
FIG. 1 depicts Video image size limitations in mobile image messaging. -
FIG. 2 depicts a system diagram for joint source-channel coding: (a) encoder; (b) decoder. -
FIG. 3 depicts a Mobile Imaging Handset Architecture. -
FIG. 4 depicts a Mobile Imaging Service Platform Architecture. -
FIG. 5 depicts a Video Codec Technology Comparison. -
FIG. 6 depicts a system diagram for improved joint source-channel coding: (a) encoder; (b) decoder. -
FIG. 7 depicts an improved mobile imaging handset platform architecture. -
FIG. 8 depicts a Video Codec Performance Comparison. -
FIG. 9 depicts an improved mobile imaging handset platform architecture. -
FIG. 10 depicts an improved mobile imaging handset platform architecture. -
FIG. 11 depicts an improved mobile imaging service platform architecture. -
FIG. 12 depicts an OTN Upgrade of deployed MMSC Video Gateway. -
FIG. 13 depicts a Self-playing video MMS eliminates need for transcoding. -
FIG. 14 depicts a reduction in complexity, cost, and number of video editing servers re to deploy media producer services. -
FIG. 15 depicts a mobile video service platform. -
FIG. 16 depicts faster, lower cost development and deployment of higher quality multimedia services. -
FIG. 17 depicts aspects of mobile video services according to aspects of the present invention. -
FIG. 18 depicts Applications to Broadband Multimedia Devices and Services according to aspects of the present invention. -
FIG. 19 depicts Implementation Options for SW Imaging Application according to aspects of the present invention. -
FIG. 20 depicts Implementation Options for HW-Accelerated Imaging Application according to aspects of the present invention. -
FIG. 21 depicts Implementation Options for Hybrid HW-Accelerated SW Imaging Application according to aspects of the present invention. -
FIG. 22 depicts an Application: Simplified Multimedia Handset Platform Architecture. -
FIG. 23 depicts Elements of the mobile video messaging demo over a GSM/GPRS network. -
FIG. 24 depicts certain MMS functionality according to aspects of the present invention. - Wavelet-Based Image Processing
- A wavelet transform may comprise the repeated application of wavelet filter pairs to a set of data, either in one dimension or in more than one. For still image compression, a 2-D wavelet transform (horizontal and vertical) can be utilized. Video codecs according to the present invention can use a 3-D wavelet transform (horizontal, vertical, and temporal). An improved, symmetrical 3-D wavelet-based video compression/decompression (codec) device is desirable to reduce the computational complexity and power consumption in mobile devices well below those required for DCT-based codecs, as well as to enable simultaneous support for processing still images and video images in a single codec. Such simultaneous support for still images and video images in a single codec may eliminate or reduce the need for separate MPEG (video) and JPEG (still image) codecs, or greatly improve compression performance and hence storage efficiency with respect to Motion JPEG codecs. An improved, symmetrical 3-D wavelet-based video processing device is also desirable to reduce the computational complexity and power consumption in MMS infrastructure equipment utilized to support automated or manual editing of user-created video, as well as database storage, search, and retrieval of user-created video.
- Aspects of the present invention comprise new methods, services and systems relating to innovative capture, compression, transmission, editing, storing and sharing video content associated with mobile devices. Aspects of the present invention may apply to telecom (both wireless and wireline providers) and internet, cable and other data and multimedia operators including fixed and mobile wireless service providers. Aspects of the present invention may provide for richer content, higher bandwidth usage and higher average revenue per user (ARPU). Mobile multimedia service (MMS) is the multimedia evolution of the text-based short message service (SMS). According to aspects of the present invention, a promising new MMS application presented is innovative video messaging and sharing, enabling a target audiences' need to communicate personal information. Mobile image messaging and sharing may require the addition of digital camera functionality (still images) and/or camcorder functionality (video images) to mobile handsets, so that subscribers can both capture (encoded) video messages that they wish to send, and play back (decode) video messages that they receive. Aspects of the present invention may also enable these functionalities in ways unavailable, if available at all, in the prior art.
- According to aspects of the present invention, mobile devices may be enabled to evolve into integrated consumer multimedia entertainment platforms. A substantial investment in industry has been directed toward technologies and platforms that enable re-packaged broadcast television programming (such as news clips, sports highlights, and special “mobisodes” of popular TV programs) and other studio-generated video content (such as film previews and music videos) to be transmitted to and viewed on mobile devices. In this later case, the mobile subscriber is exploited as a new class of video consumer. However, this later case utilizes largely video content that has been compressed in large broadcast enterprise servers. However, according to aspects of the present invention mobile operators worldwide also gain significant new opportunities to support their subscribers as media producers (as enabled by aspects of the present invention), rather than just media consumers. As enabled by aspects of the present invention, the ability to capture and share photographs and video on mobile devices with the same quality as stand-alone digital cameras and camcorders is a technical cornerstone for such new services, together with the deployment and convergence of higher speed cellular and fixed wireless data networks. Aspects of the present invention additionally enable significant reductions in the development cost and retail price of both camcorder phones and video messaging/sharing infrastructure equipment, which may be key to large scale commercial adoption of such devices and related mobile multimedia/data services, in both mature and emerging markets.
- Prior mobile image messaging/sharing services and applications are severely limited to capturing and transmitting much smaller-size and lower-frame-rate video images than those typically captured and displayed on other multimedia devices (see
FIG. 1 ), such as TVs, personal computers, digital video camcorders, and personal media players. Mobile image messaging services and applications capable of supporting VGA (or larger) video at a frame rate of 30 fps or higher, as provided by aspects of the present invention, would be preferable. - Video transmission over mobile networks is challenging in nature because of the higher data rates typically required, in comparison to the transmission of other data/media types such as text, audio, and still images. In addition, the limited and varying channel bandwidth, along with the fluctuating noise and error characteristics of mobile networks impose further constraints and difficulties on video transport. According to aspects of the present invention, various joint source-channel coding techniques can be applied to adapt the video bit stream to different channel conditions (see
FIG. 2 ). Such a joint source-channel coding approach according to aspects of the present invention may be scalable, in order to adapt to varying channel bandwidths and error characteristics. Also, according to aspects of the present invention, is supported scalability for multicast scenarios, in which different devices at the receiving end of the video stream may have different limitations on decoding computational power and display capabilities. - As shown in
FIG. 2 and according to aspects of the present invention, thesource video sequence 30 may be first source coded 32 (i.e. compressed), followed by error correction code (ECC)channel coding 34. In mobile networks, source coding typically uses such DCT-based compression techniques as, H.263, MPEG-4, or Motion JPEG. Example channel coding methods are Reed-Solomon codes, BCH codes, FEC codes, and turbo codes. The joint source and channel coded video bit stream then passes through therate controller 36 to match the channel bandwidth requirement while achieving the best reconstructed video quality. The rate controller performs discrete rate-distortion computations on the compressed video bit stream before it sends the video bit stream for transmission over thechannel 38. Due to limitations in computational power in mobile devices, typical rate controllers only consider the available channel bandwidth, and do not explicitly consider the error characteristics of the transmission channel. - It would be desirable, and is enabled by the present invention to utilize improved adaptive joint-source channel coding based on algorithms with higher computational efficiency, so that both instantaneous and predicted channel bandwidth and error conditions could be utilized in all three of the
source coder 32, thechannel coder 34, and therate controller 36 to maximize control of both the instantaneous and average quality (video rate vs. distortion) of the reconstructed video signal. - A further benefit of such an improved adaptive joint-source channel coding technique, pursuant to aspects of the present invention, is the corresponding ability of wireless carriers and MMS service providers to offer a greater range of quality-of-service (QoS) performance and pricing levels to their consumer and enterprise customers, thus maximizing the revenues generated using their wireless network infrastructure.
- Multicast scenarios require a single adaptive video bit stream that can be decoded by many users. This is especially important in modern, large-scale, heterogeneous networks, in which network bandwidth limitations make it impractical to transmit multiple simulcast video signals specifically tuned for each user. Multicasting of a single adaptive video bit stream greatly reduces the bandwidth requirements, but requires generating a video bit stream that is decodable for multiple users, including high-end users with broadband wireless or wire line connections, and wireless phone users, with limited bandwidth and error-prone connections. Due to limitations in computational power in mobile devices, the granularity of adaptive rate controllers is typically very coarse, for example producing only a 2-layer bit stream including a base layer and one enhancement layer.
- It would be desirable, and is enabled via aspects of the present invention, to utilize improved adaptive joint-source channel coding based on algorithms with higher computational efficiency, in order to enable support for a much higher level of network heterogeneity, in terms of channel types (wireless and wire line), channel bandwidths, channel noise/error characteristics, user devices, and user services.
- The addition of digital camcorder functionality to mobile handsets generally requires adding the following functions, either in hardware, software, or as a combination of hardware and software (see
FIG. 3 ): -
- imager array (typically array of CMOS or CCD pixels), with corresponding pre-amplifiers and analog-to-digital (A/D) signal conversion circuitry
- image processing functions such as pre-processing, encoding/decoding (codec), post-processing
- buffering of processed images for non-real-time transmission or real-time streaming over wireless or wire line networks
- one or more image display screens
- local image storage on built-in or removable memory.
- Using codecs based on DCT transforms, such as MPEG-4, commercially available imaging-enabled mobile handsets are limited to capturing smaller-size and lower-frame-rate video images than those typically captured and displayed on other multimedia devices, such as TVs, personal computers, digital video camcorders, and personal media players. These latter devices typically capture/display video images in VGA format (640×480 pixels) or larger, at a display rate of 30 frames-per-second (fps) or higher, whereas commercially available imaging-enabled mobile handsets are limited to capturing video images in QVGA format (320×240 pixels), QCIF format (176×144 pixels) or smaller, at a display rate of 15 fps or lower (See, e.g.,
FIG. 1 ). This reduced video capture capability is due to the excessive computational requirements, processor power consumption, and buffer memory required to complete the number, type, and sequence of computational steps associated with video compression/decompression using DCT transforms. - Using commercially available video codec and microprocessor technologies leads to very complex, power-hungry, and expensive architectures for mobile imaging handsets that target capture of VGA (or larger) video at a frame rate of 30 fps or higher. Such handset architectures require codecs that utilize a combination of both software programs and hardware accelerators running on a combination of reduced instructions set (RISC) processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), and reconfigurable processing devices (RPDs), together with larger buffer memory blocks (typical memory capacity of 1 Mbyte or more). These codec functions may be implemented using such RISC processors, DSPs, ASICs, multimedia processors, and RPDs as separate integrated circuits (ICs), or may combine one or more of the RISC processors, DSPs, ASICs, multimedia processors, and RPDs integrated together in a system-in-a-package (SIP) or system-on-a-chip (SoC).
- Codec functions running on RISC processors or DSPs are typically software routines, with the advantage that they can be modified in order to correct programming errors or upgrade functionality. The disadvantage of implementing certain complex, repetitive codec functions as software is that the resulting overall processor resource and power consumption requirements typically exceeds those available in mobile communications devices. Codec functions running on ASICs and multimedia processors are typically fixed hardware implementations of complex, repetitive computational steps, with, typically, the advantage that specially tailored hardware acceleration can substantially reduce the overall power consumption of the codec. The disadvantages of implementing certain codec functions in fixed hardware include longer and more expensive design cycles, the risk of expensive product recalls in the case where errors are found in the fixed silicon implementation, and the inability to upgrade fixed silicon functions in deployed products in the case where newly developed features are to be added to the imaging application. Codec functions running on RPDs are typically routines that require both hardware acceleration and the ability to add or modify functionality in final mobile imaging handset products. The disadvantage of implementing certain codec functions on RPDs is the larger number of silicon gates and higher power consumption required to support hardware reconfigurability in comparison to fixed ASIC implementations.
- An imaging application that reduces or eliminates complex, repetitive codec functions so as to enable mobile imaging handsets capable of capturing VGA (or larger) video at a frame rate of 30 fps with an all-software architecture would be preferable, in order to simplify the above architecture and enable handset costs compatible with high-volume commercial deployment. The present invention is the first technology to successfully accomplish and enable these objectives.
- Multimedia handsets are required not only to support picture and video messaging capabilities, but also a variety of additional multimedia capabilities (voice, music, graphics) and a variety of fixed and mobile wireless access modes, including but not limited to 2.5 G and 3 G cellular access, WiBro, HSDPA, WiFi, wireless LAN, and Bluetooth. The complexity and risk involved in developing, deploying, and supporting such products makes over-the-air (OTA) distribution and management of many functions and applications highly beneficial, in order to more efficiently deploy new revenue-generating services and applications, and to avoid costly product recalls.
- An all-SW imaging application would be preferable to enable OTA distribution and management of the imaging application by handset manufacturers, mobile operators, and other MMS service providers. Again, the present invention is the first technology to successfully enable these objectives.
- Java technology brings a wide range of devices, from servers to desktops to mobile devices, together under one language and one technology. While the applications for this range of devices differ, Java technology works to bridge those differences where it counts, allowing developers who are functional in one area to leverage their skills across a wide spectrum of devices and applications.
- First introduced to the Java community by Sun Microsystems in June 1999, J2ME (
Java 2, Micro Edition) was part of a broad initiative to better meet the diverse needs of Java developers. With theJava 2 Platform, Sun redefined the architecture of the Java technology, grouping it into three editions. Standard Edition (J2SE) offered a practical solution for desktop development and low-end business applications. Enterprise Edition (J2EE) was for developers specializing in applications for the enterprise environment. Micro Edition (J2ME) was introduced for developers working on devices with limited hardware resources, such as PDAs, cell phones, pagers, television set top boxes, remote telemetry units, and many other consumer electronic and embedded devices. - J2ME is aimed at machines with as little as 128 KB of RAM and with processors a lot less powerful than those used on typical desktop and server computers. J2ME actually consists of a set of profiles. Each profile is defined for a particular type of device—cell phones, PDAs, etc.—and consists of a minimum set of class libraries required for the particular type of device and a specification of a Java virtual machine required to support the device. The virtual machine specified in any J2ME profile is not necessarily the same as the virtual machine used in
Java 2 Standard Edition (J2SE) andJava 2 Enterprise Edition (J2EE). - It is very difficult to define a single J2ME technology that would be optimal, or even close to optimal, for all of the devices listed above, due to the differences in processor power, memory, persistent storage, and user interface. To address this problem, Sun divided and then subdivided the definition of devices suitable for J2ME into sections. First, Sun divided devices into two broad categories based on processing power, memory, and storage capability, with no regard for intended use. The company then defined a stripped-down version of the Java language that would work within the constraints of the devices in each category, while still providing at least minimal Java language functionality.
- Next, Sun identified within each of these two categories classes of devices with similar roles—so, for example, all cell phones fell within one class, regardless of manufacturer. With the help of its partners in the Java Community Process (JCP), Sun then defined additional functionality specific to each class of devices.
- The first division created two J2ME configurations: Connected Device Configuration (CDC) and Connected, Limited Device Configuration (CLDC). A configuration may be a Java virtual machine (JVM) and a minimal set of class libraries and APIs providing a run-time environment for a select group of devices. A configuration may specify a least common denominator subset of the Java language, one that fits within the resource constraints imposed by the family of devices for which it was developed. Because there is such great variability across user interface, function, and usage, a typical configuration does not define such important pieces as the user interface toolkit and persistent storage APIs. The definition of that functionality belongs, instead, to what is called a profile.
- A J2ME profile may be a set of Java APIs specified by an industry-led group that is meant to address a specific class of device, such as pagers and cell phones. Each profile is built on top of the least common denominator subset of the Java language provided by its configuration, and is meant to supplement that configuration. Two profiles important to mobile handheld devices are: the Foundation profile, which supplements the CDC, and the Mobile Information Device Profile (MIDP), which supplements the CLDC. More profiles are in the works, and specifications and reference implementations continue to be developed and released.
- The Java Technology for the Wireless Industry (JTWI) specification, JSR 185, defines the industry-standard platform for the next generation of Java technology-enabled mobile phones. JTWI is defined through the Java Community Process (JCP) by an expert group of leading mobile device manufacturers, wireless carriers, and software vendors. JTWI specifies the technologies that must be included in all JTWI-compliant devices: CLDC 1.0 (JSR 30), MIDP 2.0 (JSR 118), and WMA 1.1 (JSR 120), as well as CLDC 1.1 (JRS 139) and MMAPI (JSR 135) where applicable. Two additional JTWI specifications that define the technologies and interfaces for mobile multimedia devices are JSR-135 (“Mobile Media API”) and JSR-234 (“Advanced Multimedia Supplements”).
- The JTWI specification raises the bar of functionality for high-volume devices, while minimizing API fragmentation and broadening the substantial base of applications that have already been developed for mobile phones. Benefits of JTWI include:
-
- Interoperability: The goal of this effort is to deliver a predictable environment for application developers, and a deliverable set of capabilities for device manufacturers. The intent is that both benefit by adopting the JTWI standard: manufacturers benefit from a broad range of compatible applications, software developers benefit from a broad range of devices that support their applications.
- Clarification of security specification: The JSR 185 specification introduces a number of clarifications for untrusted applications with regard to the “Recommended Security Policy for GSM/UMTS-Compliant Devices” defined in the MIDP 2.0 specification. It extends the base MIDlet suite security framework defined in MIDP 2.0.
- Road map: A key feature of the JTWI specification is the road map, an outline of common functionality that software developers can expect in JTWI-compliant devices. January 2003 saw the first in a series of road maps expected to appear at six- to nine-month intervals, which will describe additional functionality consistent with the evolution of mobile phones. The road map enables all parties to plan for the future with more confidence: carriers can better plan their application deployment strategy, device manufacturers can better determine their product plans, and content developers can see a clearer path for their application development efforts. Carriers in particular will, in the future, rely on a Java VM to abstract/protect underlying radio/network functions from security breaches such as viruses, worms, and other “attacks” that currently plaque the public Internet.
- A Java-based imaging application would be preferable for “write-once, run-anywhere” portability across all Java-enabled handsets, for Java VM security and handset/network robustness against viruses, worms, and other mobile network security “attacks”, and for simplified OTA codec and application download procedures. Such a Java-based imaging application should conform to JTWI specifications JSR-135 (“Mobile Media API”) and JSR-234 (“Advanced Multimedia Supplements”). Aspects of the present invention provide these advantages.
- Key components of a mobile imaging service platform architecture according to aspects of the present invention (see
FIG. 4 ) may include: -
-
Mobile Handsets 60 - Mobile Base stations (BTS) 62
- Base station Controller/Radio Network Controller (BSC/RNC) 64
- Mobile Switching Center (MSC) 66
- Gateway Service Node (GSN) 68
- Mobile Multimedia Service Controller (MMSC) 70
-
- Typical functions included in the MMSC according to aspects of the present invention (see
FIG. 4 ) include: -
-
Video gateway 72 -
Telco server 74 -
MMS applications server 76 -
Storage server 78
-
- The video gateway in an MMSC, according to aspects of the present invention, may serve to transcode between the different video formats that are supported by the imaging service platform. Transcoding is also utilized by wireless operators to support different voice codecs used in mobile telephone networks, and the corresponding voice transcoders are integrated into the RNC. Upgrading such a mobile imaging service platform with the architecture shown in
FIG. 4 if found in prior architectures, would typically require deploying new handsets, and manually adding new hardware to the MMSC video gateway. In some mobile video messaging and sharing applications, it may be desirable to eliminate the cost and complexity associated with transcoding. One aspect of the current invention is the ability to embed a software decoder with each transmitted video stream, enabling “self-playing” functionality on common handset and PV video players. - The MMS applications servers in an MMSC may support applications such as automated or manual editing of user-created video, as well as database storage, search, and retrieval of user-created video. The computational complexity required to implement such functions requires specialized servers to be installed by mobile operators, with the corresponding video processing functions typically requiring expensive and high-power application-specific integrated circuits (ASICs) and digital signal processors (DSPs), rather than simpler SW applications running on less-expensive and lower-power CPU chips used in standard personal computers (PCs) and servers.
- As is enabled by aspects of the present invention, an all-software mobile imaging applications service platform according to aspects of the present invention, would be preferable, in order to support automated OTA upgrade of deployed handsets, automated OTN upgrade of deployed MMSCs, and support for mobile video applications using standard PCs and servers. A Java implementation of the mobile handset imaging application, according to aspects of the present invention, may be preferable in terms of improved handset/network robustness against viruses, worms, and other “attacks”, allowing mobile network operators to provide the quality and reliability of service required by national regulators.
- The deployment of mobile video messaging and sharing services has exposed fundamental limitations of video compression technologies.
- On the one hand, such mobile video services are being launched into a market that now equates video with home cinema quality broadcast—full size image formats such as VGA at 30 frames per second. On the other hand, processing of such large volumes of data using existing video technologies originally developed for broadcasting and streaming applications greatly exceeds the computing resources and battery power available for real-time video capture (encoding) in mobile handsets. Broadcast and streaming applications rely on the encoding of video content in a studio environment, where high-complexity encoders can be run on computer workstations. Since video messages must be captured in real time in the handset itself, they are limited to much smaller sizes and much lower frame rates.
- As a result, today's mobile video services are primitive; pictures are small (QCIF, QVGA) and choppy (15 fps or less) in comparison to those that subscribers have long come to expect from the digital camcorders whose functionality video phones have been positioned to replicate. The primitive video image quality offered to mobile subscribers today also falls far short of the crisp high-definition video featured in the industry's lifestyle advertising. Mobile subscribers are demanding full VGA, 30 fps performance (i.e. just like their camcorder) before they will widely adopt and pay premium pricing for camcorder phones and related mobile video services.
- Even after highly expensive and time-consuming development programs, competing video codec providers apart from the present inventors can still only offer complex hybrid SW codec+HW accelerator solutions in attempts to provide
VGA 30 fps performance, with overall cost and power consumption that far exceed commercial business requirements and technology capabilities. Handsets are thus limited to small choppy images, or expensive power-hungry architectures. Service deployment is too expensive, and quality of service is too low, to attract mass-market adoption. - Upgrading MMSC infrastructure is also costly if new or specialized hardware is required. An all-SW applications and service platform would be preferable in order to enable automated OTA upgrade of handsets, OTN upgrade of MMSC video gateways, and support for mobile video applications using standard PCs and servers. The need for transcoding between different video formats also adds additional cost and complexity.
- The solution to this problem, particularly according to aspects of the present invention, is a much lower complexity imaging application (codec) that can be implemented as an all-SW application on mobile handsets, reducing the complexity of the handset architecture and the complexity of the mobile imaging service platform architecture. According to certain aspects of the present invention, an all-SW video codec solution substantially reduces or eliminates baseband processor and video accelerator costs and requirements in multimedia handsets. Combined with the ability to install the codec post-production via OTA download, this all-SW solution substantially reduces the complexity, risk, and cost of both handset development and video messaging service architecture and deployment. SW video transcoders and editing, storing, searching, retrieval applications according to the present invention enable automated over-the-network (OTN) upgrade of deployed MMS control (MMSC) infrastructure, as well as the use of standard PCs and servers to run such applications. Additionally, the present invention wavelet transcoders provide carriers with complete interoperability between the wavelet video format and other standards-based and proprietary video formats. The present invention also allows a software decoder to be embedded with each transmitted video stream, enabling “self-playing” functionality on common handset and PV video players, and eliminating the cost and complexity of transcoding altogether. The present invention's all-SW video platform allows rapid deployment of new MMS services, also parts of embodiments of the present invention, that leverage processing speed and video production accuracy not available with other existing technologies. Such new MMS services are themselves aspects of the current invention. The present invention's wavelet codecs are also unique in their ability to efficiently process both still images and video, and can thus replace separate MPEG and JPEG codecs with a single lower-cost and lower-power solution that can simultaneously support both mobile picture-mail and video-messaging services.
- In the present description, certain aspects, figures, or embodiments are decribed as “Droplet” aspects or embodiments. In the present application, “Droplet” should be understood to refer to embodiments of the present invention.
- Aspects of the present invention utilize 3-D wavelet transforms in video compression/decompression (codec) devices with much lower computational complexity than DCT-based codecs (
FIG. 5 provides a comparison of the relative computational requirements of a traditional DCT encoder technology and exemplary technologies of the present invention). The application of a wavelet transform stage also enables design of quantization and entropy-coding stages with greatly reduced computational complexity. Further advantages of the 3-D wavelet codecs of the present invention for mobile imaging applications, devices, and services include: -
- Symmetric, low-complexity video encoding and decoding
- Lower processor power requirements for both SW and HW codec implementations
- All-software encoding and decoding of VGA (or larger) video at a frame rate of 30 fps (or higher) with processor requirements compatible with existing commercial mobile handsets, both as native code and as a Java application
- Lower gate-count ASIC cores for SoC integration
- Lower buffer memory requirements
- Single codec supports both still images (˜JPEG) and video (˜MPEG)
- Simplified video editing (cuts, inserts, text overlays due to shorter group of pictures (GOP)
- Simplified synchronization with voice codecs, due to shorter GOP
- Low latency for enhanced video streaming, due to shorter GOP
- Fine grain scalability for adaptive rate control, multicasting, and joint source-channel coding
- Low-complexity performance scaling to emerging HDTV video formats
- Compact SW decoder (for example, such as less than 40 kB in size) can be integrated with each transmitted video stream to enable “self playing” video messages compatible with common handset and PC video players.
- The above advantages are achieved by aspects of the present invention including:
- Application of wavelet transforms using short dyadic integer filter coefficients in the lifting structure: in certain embodiments the Haar, 2-6, and 5-3 wavelets and variations of them are used. These can require only adds, subtracts, and small fixed shifts—no multiplication or floating-point operations are needed.
- Lifting Scheme computation: These filters can be computed using the Lifting Scheme which allows in-place computation. This minimizes use of registers and temporary RAM locations, and keeps references local for highly efficient use of caches.
- Wavelet transforms in pyramid form with customized pyramid structure: Certain embodiments of the present invention compute each level of the wavelet transform sequence on half of the data resulting from the previous wavelet level, so that the total computation is almost independent of the number of levels. Aspects of the present invention customize the pyramid to leverage the advantages of the Lifting Scheme above and further economize on register usage and cache memory bandwidth.
- Block structure: In contrast to most wavelet compression implementations, aspects of the present invention may divide the picture into rectangular blocks and processes each block separately from the others. This allows memory references to be kept local and to do an entire transform pyramid with data that remains in the processor cache, saving a significant amount of data movement within most processors. The present block structure is particularly helpful in HW embodiments as it avoids the requirement for large intermediate storage capacity in the signal flow.
- Block boundary filters: the present invention may also use modified filter computations at the boundaries of each block that avoid sharp artifacts as set out in U.S. patent application Ser. No. 10/418,363, incorporated herein by reference.
- Chroma temporal removal: aspects of the present invention may also avoid processing the chroma-difference signals for every field, instead using a single field of chroma for a GOP as set out in U.S. patent application Ser. No. 10/447,514, incorporated herein by reference.
- Temporal compression using 3D wavelets: Certain embodiments of the present invention may not use the very expensive motion-search and motion-compensation operations of conventional video compression methods such as MPEG. Instead those embodiments compute a field-to-field temporal wavelet transform. This is much less expensive to compute. Also sometimes used are short integer filters with the Lifting Scheme in this aspect.
- Dyadic quantization: In certain embodiments of the present invention, the quantization step of the compression process may be accomplished using a binary shift operation uniformly over a range of coefficient locations. This avoids the per-sample multiplication or division required by conventional quantization.
- Piling: In certain embodiments of the present invention, the amount of data to be handled by the following entropy coder by doing run-of-zeros conversion is reduced. In certain embodiments, the methods and inventions disclosed in U.S. patent application Ser. No. 10/447,455 are utilized for a counting runs of zeros on parallel processing architectures.
- Cycle-efficient entropy coding: In certain embodiments of the present invention, the entropy coding step of the compression process is accomplished using techniques that combine the traditional table lookup with direct computation on the input symbol. Because the symbol distribution has been characterized, such simple entropy coders as Rice-Golomb or exp-Golomb or Dyadic Monotonic can be used. The choice of entropy coder details will often vary depending on the processor platform capabilities.
- The fine grain scalability of the wavelet-based codec according to aspects of the present invention enables improved adaptive rate control, multicasting, and joint source-channel coding. The reduced computational complexity and higher computational efficiency of the present wavelet algorithms allows information on both instantaneous and predicted channel bandwidth and error conditions to be utilized in all three of the source coder, the channel coder, and the rate controller to maximize control of both the instantaneous and average quality (video rate vs. distortion) of the reconstructed video signal (see
FIG. 6 ). The improved adaptive joint-source channel coding technique of the present invention allows wireless carriers and MMS service providers to offer a greater range of quality-of-service (QoS) performance and pricing levels to their consumer and enterprise customers. Utilizing improved adaptive joint-source channel coding based on algorithms with higher computational efficiency enables support for a much higher level of network heterogeneity, in terms of channel types (wireless and wire line), channel bandwidths, channel noise/error characteristics, user devices, and user services. -
FIG. 7 illustrates an improved mobile imaging handset platform architecture according to aspects and embodiments of the present invention. The imaging application according to aspects of the present invention is implemented as an all-software application running as native code or as a Java application on a RISC processor. Acceleration of the Java code operation may be implemented within the RISC processor itself, or using a separate Java accelerator IC. Such a Java accelerator may be implemented as a stand-alone IC, or this IC may be integrated with other functions in either a SIP or SoC. - The improved mobile imaging handset platform architecture illustrated in
FIG. 7 eliminates the need for separate DSP, ASIC, multimedia processor, or RFD processing blocks for the mobile imaging application, (as would be required in prior devices or systems) and also greatly reduces the buffer memory requirements for image processing in the mobile handset. -
FIG. 8 shows the reduction in computational requirements forfull VGA 30 fps video encoding provided by aspects of the current invention, in comparison to current state-of-the-art industry solutions reached after the filing date of the present application's priority filing based upon MPEG-4 and H-264 video codecs. -
FIG. 9 shows one implementation of aspects of the current invention on a commercial mobile GSM camcorder phone platform. Whereas the existing GSM baseband/multimedia SoC (Texas Instruments OMAP 850 shown inFIG. 9 ) requires a HW accelerator, a DSP, and a RISC processor for QCIF/15 fps camcorder functionality, the present invention provides VGA/30 fps camcorder functionality on this platform using only SW running on the RISC processor without the need of a HW accelerator or a DSP. -
FIG. 10 shows one implementation of aspects of the current invention on a commercial mobile CDMA camcorder phone platform. Whereas the existing CDMA baseband/multimedia SoC (Qualcomm MSM6500 shown inFIG. 10 ) requires a HW accelerator, a DSP, and a RISC processor for QCIF/15 fps camcorder functionality, the present invention provides VGA/30 fps camcorder functionality on this platform using only SW running on the RISC processor without the HW accelerator or DSP. - Components of an improved mobile imaging service platform architecture according to embodiments of the present invention (see
FIG. 11 ) may include: -
-
Mobile Handsets 160 - Mobile Base stations (BTS) 162
- Base station Controller/Radio Network Controller (BSC/RNC) 164
- Mobile Switching Center (MSC) 166
- Gateway Service Node (GSN) 168
- Mobile Multimedia Service Controller (MMSC) 170
- Imaging
Service Download Server 171
-
- Typical functions included in the MMSC (see
FIG. 11 ) may include: -
-
Video Gateway 172 -
Telco Server 174 -
MMS Applications server 176 -
Storage Server 178
-
- According to aspects of certain embodiments of the present invention, certain steps involved in deploying the improved imaging service platform may include:
-
Step 1. - Signal network that Video Gateway Transcoder application and/or video messaging/sharing applications are available for updating deployed MMSCs. The update can be installed via automated OTN deployment or via manual procedures.
-
Step 2. - Install and configure Video Gateway Transcoder SW application and/or video messaging/sharing SW applications via automated OTN deployment or via manual procedures (see
FIG. 12 ). -
Step 3. - Signal subscriber handset that Mobile Video Imaging Application is available for download and installation.
-
Step 4. - If accepted by subscriber, and transaction settlement is completed successfully, download and install Mobile Video Imaging Application.
- Step 5.
- Signal network that handset upgrade is complete. Activate service and related applications. Update subscriber monthly billing records to reflect new charges for Mobile Video Imaging Application.
- According to certain aspects of the present invention,
FIG. 13 shows “self-playing” video MMS functionality achieved by integrating the SW decoder with the transmitted video stream. - According to certain aspects of the present invention,
FIG. 14 shows the reduction in complexity, cost, and number of video application servers required to deploy media producer services such as automated or manual editing of user-created video, as well as database storage, search, and retrieval of user-created video. - According to certain aspects of the present invention,
FIG. 15 shows the functional elements of a video messaging/sharing/calling platform incorporating the improved wavelet-based codec/camcorder application, improved joint source channel coding, and improved video editing and database storage, search, and retrieval. - According to certain aspects of the present invention,
FIG. 16 shows the benefits in terms of faster, lower cost development and deployment of higher quality multimedia handsets & services, including the ability to deploy an innovative personal multi-media market place platform in which users can preview, share, buy, and sell “soft” copies (download) or “hard” copies (DVD) of user-created audio/video content. The present invention also allows for more efficient video “tagging” for database indexing and network (RSS) feeds, and supports interfaces to existing web-based market places such as E-bay, Google, Yahoo, Microsoft, and other portals. - According to certain aspects of the present invention,
FIG. 17 shows several innovative new mobile video services based on the improved wavelet-based codec/camcorder application, improved joint source channel coding, and improved video editing and database storage, search, and retrieval. - According to certain aspects of the present invention,
FIG. 18 shows applications of the above video messaging/sharing/calling platform incorporating the improved wavelet-based codec/camcorder application, improved joint source channel coding, and improved video editing and database storage, search, and retrieval, to deploy new video services on fixed wireless, mobile wireless, and wireline networks, as well as “converged” networks combining elements of fixed wireless, mobile wireless, and wireline architectures. - Aspects of the present invention, with their improved wavelet-based mobile video imaging application, joint source-channel coding, handset architecture, and service platform architecture achieve goals of higher mobile video image quality, lower handset cost and complexity, and reduced service deployment costs.
- Various embodiments of aspects of the present invention provide enhancements to the mobile imaging handset architecture. For example, several implementation options can be considered for the all-SW wavelet-based imaging application (see
FIG. 19 ). The imaging application can be installed via OTA download (400 a, 400 b, 400 c) to the baseband multimedia processing section of thehandset 402 a, to aremovable storage device 402 b, or to theimaging module 402 c. Where desirable, the imaging application can also be installed during manufacturing or at point-of-sale to the baseband multimedia processing section of the handset, to a removable storage device, or to the imaging module. Additional implementation options are also possible as mobile device architectures evolve. - According to aspects of the present invention, performance of the mobile imaging handset may be further improved, and costs and power consumption may be further reduced, by accelerating some computational elements via HW-based processing resources in order to take advantage of ongoing advances in mobile device computational HW (ASIC, DSP, multimedia processor, RPD) and integration technologies (SoC, SIP). Several all-HW options can be implemented for integrating these hardware-based processing resources in the handset (see
FIG. 20 ), including the baseband multimedia processing section of the handset, a removable storage device, or the imaging module. - As shown in
FIG. 21 , hybrid architectures offered by aspects of the present invention for the imaging application may offer enhancements by implementing some computationally intensive, repetitive, fixed functions in HW, and implementing in SW those functions for which post-manufacturing modification may be desirable or required. - According to aspects of the current invention,
FIG. 22 shows potential simplifications in mobile camcorder device architecture, deployment, and maintenance. - The all-SW imaging solution of the present invention substantially reduces baseband processor and video accelerator costs and requirements in multimedia handsets. Combined with the ability to install and maintain the codec post-production via OTA download, this all-SW solution can substantially reduce the complexity, risk, and cost of both handset development and video messaging service deployment.
- The present invention provides mobile operators with the first mobile video messaging and sharing platform that delivers the video quality, mobile handset price-point, and service deployment costs required for mass-market adoption by consumer and enterprise customers. The present invention provides the first all-SW camcorder phone application capable of real-time capture of full (VGA)-size images (640×480 pixels) at 30 frames per second (fps), using only according to certain aspects and embodiments of the present invention standard RISC processors already incorporated in the vast majority of multimedia handsets. For mobile carriers, the present invention's low-complexity video processing and distribution technologies can be integrated into a powerful new all-software platform that enables turnkey deployment using existing mobile handsets and mobile Multimedia Messaging Service (MMS) infrastructure. Complementing the above all-SW mobile camcorder application, aspects of the present invention's content management platform provide carriers with modules for integrating compressed images and videos, according to the present technology, together with sounds and text into complete mobile multimedia messages and “ring-tones”, along with on-the-fly editing, thumbnail previews, multimedia mailboxes, on-line repository, sharing, and marketing services, and subscription management.
- Example 1, below, describes the components, setup, and operation of an introductory demonstration of the functionality and benefits provided by an embodiment of aspects of the present invention's software-only mobile video messaging platform. The demo utilizes commercially available GSM/GPRS multimedia handsets, and was designed to operate over any commercial GSM/GPRS network. The demonstration operated very successfully. The demo can also be readily adapted to utilize CDMA handsets, and to operate over any commercial CDMA network. The demo in Example 1 runs a demo and set of files code named “Droplet” and so labeled in Example 1.
- As illustrated in
FIGS. 23 and 24 , the demo includes the following five elements: -
- 1. Downloading the present invention's all-SW “DTV” video codec application over-the-air (OTA) from an MMS service provider's server, and installing in a multimedia handset.
- 2. Using the DTV video codec to compress high-quality (VGA, 30 fps) video messages in the multimedia handset
- 3. Sending the compressed video messages (the present invention's DTV format) to an MMS server, either in full VGA/30 fps resolution for forwarding/playback on a computer/TV, or scaled down to lower resolution (QCIF/15 fps) for forwarding/playback on another multimedia handset.
- 4. Notification by the MMS server, via an email alert to a networked computer or an SMS alert to another handset, that the compressed video images are available for download.
- 5. Downloading the compressed video messages, together with the present invention's DTV decoder, for “self-playing” function, i.e. decoding on a networked computer (VGA/30 fps) or on another handset (QCIF/15 fps) and playback using common video players already installed on the PC or handset.
- 2.1 Handsets
- For the version of the demo described in this Example 1, the two commercially available GSM/GPRS multimedia handsets described below are used. Many other comparable multimedia handsets can also be utilized.
- 2.1.1 T-Mobile MDA-II PDA-Phone (Manufactured by HTC)
-
- Needs to be operational on a local GSM/GPRS network
- Data/internet access service should be activated
- MDA-II phone cradle with USB 2.0 cable to connect the handset to a PC
- OS: Windows Mobile 2003 Pocket PC with Phone Edition
- 2.1.2 O2 Xphone Smartphone (Manufactured by HTC)
-
- Needs to be operational on a local GSM/GPRS network
- Data/internet access service should be activated
- USB 2.0 cable to connect the handset to a PC
- OS: Windows Mobile 2003 Smartphone Edition
- 2.2 Computers
- 2.2.1. PC (Laptop or Desktop)
- For the version of the demo described in Example 1, a laptop computer is utilized, with the following configuration:
-
- Model: Sony Vaio PCG-K33P
- Processor/Memory: 3.06
GHz Pentium 4, 512 MB RAM - Graphics: ATI Radeon IGP 345M with 64 MB RAM
- Storage: 60 GB
- I/O: one 1394 (Firewire) port, two USB 2.0 ports
- OS: Windows XP
- Pre-installed video application: Windows Movie Maker
- 2.2.2. Remote Server
- For the version of the demo described in Example 1, a UNIX-based remote server is used to perform the following MMS functions:
- Store DTV codec for download by MDA-II handset
-
- Store DTV player application for download by Xphone
- Receive compressed video messages from MDA-II handset for storage/forwarding
- Send SMS message notifying Xphone that video message is available for download
- Enable Xphone to access (via browser) both the DTV player application and QCIF/15 fps video messages for download
- 2.3 Video Capture
- For the version of the demo described in Example 1, the following commercial camcorder is utilized to provide an external source of high-quality video input sequences for compression by the present invention's DTV codec running on the handsets:
-
- Canon ZR65 DV Camcorder (with 1394 Firewire cable to connect camcorder to PV)
- Use of the external camcorder, rather than the much lower performance imager modules currently available in most commercial camcorder phones, enables a much more compelling demonstration of the benefits of the present invention's DTV codec and mobile video messaging capabilities.
- The compressed DV video files captured by the camcorder are first converted into decompressed UYVY video format in the PC, and then input to the MDA-II handset for encoding/compression by the present invention's DTV codec. UYVY is a typical video format that would be input to the video codec in a multimedia handset.
- 3.1 Configuring the PC
- 3.1.1 Downloading the Droplet Demo Package According to Aspects of the Present Invention
-
- Open the URL http://droplet-tech.com/demo_partner_access.html in the PC's web browser.
- Select the link “Demo Package—January 2005” (WinZip archive).
- Click on this link to start downloading the demo package to the PC.
- Place the zip file into a folder of your choosing
- Unzip the file and you should see the following:
- Demo package “read me” document (Demo_package_readme.html)
- The following directories:
- MDA_DTV: MDA-II client UI applications
- Xphone_DTV: Xphone UI client applications
- PC_player: files for playing DTV files on the PC
- Virtual_Dub1.6.3: PC software app for converting between different video formats
- MMS_server: sample monitoring script for the server
- Canon_driver: camcorder driver software
- PHMRegEditor: Registry editor for installtion on MDA-II and Xphone
- Ewesoft: JVM for the Xphone
- 3.1.2 Downloading the J9 JVM (Java Virtual Machine) Installation Package
-
- Open the URL http://droplet-tech.com/ftp_access.html in the PC's web browser.
- Select the link “J9_JVM” (WinZip archive).
- Click on this link to start downloading the J9 JVM package to the PC.
- Put the zip file “weme57prod_ppwm—1.zip” into a folder of your choosing.
- 3.1.3 Downloading and Installing DirectX 9.0 SDK (Utilized by DTV PC Player)
-
- This free SDK can be downloaded from the Microsoft website:
- http://msdn.microsoft.com/library/default.asp?url=/downloads/list/directx.asp
- Note that this is a very large download (˜230 MB)
- Launch DirectX Utilities->GraphEdit function to verify successful installation.
- 3.1.4 Installing the Present Invention's Droplet DTV PC Player on PC
-
- The software package can be found in the demo package under the PC_player directory. The file is called PC_player.zip.
- Unzip the package to the “C:” drive on the test PC as the folder “C:\DTV_PCplayer_Demo\”. If the actual path is different, edit RegisterFilter.bat and UnRegisterFilter.bat.
- Open the folder and register the DirectShow filter by double-clicking the batch file “RegisterFilter.bat”.
- 3.1.5 Downloading and Installing Virtual Dub on PC
- In the present version of the demo, Virtual Dub is used to convert the compressed DV video files, as captured by the camcorder, into decompressed UYVY video format in the PC. These decompressed video files are then input to the MDA-II handset for encoding/compression by the present invention's DTV codec. UYVY is a typical video format that would be input to the video codec in a multimedia handset.
-
- The software package is found in the demo package under the Virtual_Dub directory.
- Alternatively, the software package can also be found at the following URL: http://jaist.dl.sourceforge.net/sourceforge/virtualdub/VirtualDub-1.6.3.zip
- In the Virtual_Dub directory is a file called VirtualDub.exe. This is selected and the application is verified to run.
- 3.1.6 Installation of the Canon ZR65 Camcorder Driver on PC
-
- The Canon driver package is found in the demo package under the Canon_driver directory.
- Alternatively, the driver can also be found at http://www.canon.com.
- The Canon_driver directory is opened and the subfolder “ZR65WI503EN” then subfolder “English” are selected and then the “SETUP.EXE” executable program is run to install the Canon camcorder driver.
- Note: If another type of camcorder is used, the appropriate driver must be installed on the PC. The camcorder must be capable of DV video capture while connected to the PC.
- 3.2 Configuring the Remote MMS Server
- In the demo, the remote MMS server functions both as an FTP server (to enable download of video codec files to the handsets, and network storage of video files from the recording handset), and as a mail server (to enable email/SMS notification and download of video messages by networked computers and other handsets). Functionally, the server must be able to send SMS messages, in order to enable SMS notification to other handsets of pending video messages.
-
- Configure the FTP site to have the following directories:
- public_html
- Pre-install the DTV codec (file DtvMDADemo.exe) on the server, for subsequent download by the MDA-II handset. This file should be transferred in binary mode to the FTP server at the top level.
- Pre-install the DTV decoder (file DtvXphoneDemo.exe) on the server, for subsequent download by the Xphone. This file should be transferred in binary mode to the FTP server at the top level.
- A sample monitoring script is in the Droplet demo package under the MMS_server directory (filename: monitor.php).
- This monitoring script is referenced in crontab on Unix based servers. This script will monitor every minute for the presence of new files (ending in .Ink) in the ftp:/public_html directory.
- If the user chooses to use the above sample monitoring script on a server other than the Droplet demo server, it must be customized for the local server/network environment.
- Configure the FTP site to have the following directories:
- 3.3 Configuring the MDA-II
- Connect the MDA-II to the PC (pre-configured as described in Section 3.1) via the cradle/USB cable.
- 3.3.1 Modifying the Registry Entries for Stable GPRS Connection
- To ensure a stable GPRS connection, the timeout period is increased to greater than the default setting of 60 sec. The handset manufacturer (HTC) has provided the recommended registry changes. If there is no Registry Editor installed on the device, first install the registry editor included in Droplet's Demo package under the “PHMRegEditor” directory.
-
- Place the file PHMRegEdit.msi in the Temp directory on your PC
- Place the CAB files regedit.Mrln_ARM.CAB in the same directory on the PC
- Run PHMRegEdit.msi by double-clicking on it and follow the instructions
- Synchronize with the MDA-II
- Copy the file regedit.Mrln_ARM.CAB to the Windows directory on the MDA-II
- Double-click on that file in the MDA-II and it will install the CAB files appropriately
- Run PHMRegEdit.msi and follow the instructions.
- The resulting program PHMEditor is installed on the MDA-II in the directory \Program Files\PHM Tools
- Start the program regedit (the registry editor) from the above directory
- Select HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ConnMgr\Planner\Setting s
- Change the setting for CacheTime to 300 (this is 5 min. timeout period)
- Change the setting for SuspendResume to ˜GPRS! (allows for no timeout)
- Exit the registry editor.
- Perform a soft reset to allow the modified settings to take effect
- 3.3.2 Installing IBM J9 JVM
-
- Unzip the “weme57prod_pp_wm—1.zip” file that was previously downloaded on the PC (Section 3.1.2). There are 5 files.
- inst_pp_wm.html
- readme_pp_wm.html
- weme-wm2003-arm-ppro10-5.7.1-P-20040723-1833.bin
- weme-wm2003-arm-ppro10-5.7.1-P-20040723-1833.exe
- weme57_ppc_pp—1.pdf
- Open file inst_pp_wm.html. Read and follow the installation instructions.
- Unzip the “weme57prod_pp_wm—1.zip” file that was previously downloaded on the PC (Section 3.1.2). There are 5 files.
- 3.3.3 Installing the MDA-II Handset UI Application
- Note that this application will be found in the files that were already unzipped on the PC as described in Section 3.1.1.
-
- From the demo package, the directory MDA_DTV contains a subfolder called “mms_client”. There should be 3 files:
- “application.properties”
- “Droplet.jar”
- “Droplet.Ink”
- Copy the files “application.properties” and “Droplet.jar” from the PC to the MDA directory: \ProgramFiles\J9\PPRO10\examples.
- Copy the file “Droplet.lnk” to the MDA directory \Windows\Start Menu.
- To verify the MDA-II has been configured correctly, click on the Start menu. An icon called Droplet should now be available.
- To verify the functionality, select the Droplet icon and a new window pops up. It will have 3 buttons:
- Download/install codec
- Capture Video
- Send Video Message
- From the demo package, the directory MDA_DTV contains a subfolder called “mms_client”. There should be 3 files:
- Note: source code for the MDA-II Handset UI Application is available under the directory MDA_DTV\mms_client_src as a reference.
- The MDA-II has now been configured for the demo.
- 3.3.4 Installing the Pictpocket Cinema Video Player (Optional)
- This step is optional, since the default video player on the MDA-II can view the decoded Droplet video file. By installing a third party video player, this demonstrates that the present invention's decoded video files are compatible with multiple mobile device video players.
-
- On the MDA-II handset, the “Pictpocket Cinema” application from Digisoft has been installed.
- A 14-day trial version of Pictpocket Cinema can be downloaded from the vendor's website: www.digisoftdirect.com.
- 3.4 Configuring the Xphone
- Connect the Xphone to the PC via the USB cable.
- 3.4.1 Modifying the Registry Entries for Stable GPRS Connection
- To ensure a stable GPRS connection, the timeout period needs to be increased to greater than the default setting of 60 sec. The handset manufacturer (HTC) has provided the recommended registry changes. If there is no Registry Editor installed on the device, first install the registry editor included in Droplet's Demo package under the “PHMRegEditor” directory.
-
- Place the file PHMRegEdit.msi in the Temp directory on your PC
- Place the CAB files regedit.Mrln_ARM.CAB in the same directory on the PC
- Run PHMRegEdit.msi by double-clicking on it and follow the instructions
- Synchronize the PC with the Xphone
- Copy the file regedit.Mrln_ARM.CAB to the Windows directory on the Xphone
- Double-click on that file in the Xphone and it will install the CAB files appropriately
- Run PHMRegEdit.msi and follow the instructions.
- The resulting program PHMEditor will be installed on the Xphone in the directory \Program Files\PHM Tools
- Start the program regedit (the registry editor) from the above directory
- Select HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ConnMgr\Planner\Setting s
- Change the setting for CacheTime to 300 (this is 5 min. timeout period)
- Change the setting for SuspendResume to ˜GPRS! (should allow for no timeout)
- Exit the registry editor.
- Perform a soft reset to allow the modified settings to take effect
- 3.4.2 Installing Ewesoft JVM
-
- From the Droplet demo package, in the directory Ewesoft will be a file called Ewe143-CAB-SmartPhone.zip. Unzip this file on the PC. Use the file called Ewe-SmartPhone2003.arm.CAB.
- Copy the Ewe-SmartPhone2003.arm.CAB file to the Xphone directory called: Storage\windows\Start Menu\Accessories
- Go to Start->Accessories. The CAB file should be displayed as a menu item. Select the CAB file and the VM will then be installed
- The installation will create a new Ewe folder that will appear in the Start menu. Within that folder you will find the Ewe VM itself and a “Solitaire” demo application with which to verify correct installation.
- 3.4.3 Installing the Xphone Handset UI Application
-
- From the Droplet demo package, in the directory Xphone_DTV will be a subfolder called “mms_client”. There should be 3 files.
- “application.properties”
- “Droplet.jar”
- “Droplet.Ink”
- Copy the files “application.properties” and “Droplet.jar” from the PC to the Xphone directory: /Storage/windows/StartMenu/Ewe/.
- Copy the file “Droplet.lnk” to the Xphone directory: /Storage/windows/StartMenu/.
- To verify the Xphone has been configured correctly, click on the Start menu. An icon called Droplet should now be available
- To verify the functionality, select the Droplet icon and a new window pops up. It will have 3 buttons:
- Download/install codec
- Capture Video
- Send Video Message
- From the Droplet demo package, in the directory Xphone_DTV will be a subfolder called “mms_client”. There should be 3 files.
- The Xphone has now been configured for the demo
- 4.1 Downloading and Installing the Video Codec on the MDA-II
-
- From the MDA, click on the Start menu and a Droplet icon should appear on the list.
- Select the Droplet icon and a new window pops up. It will have 3 button selections:
- Download/Install codec
- Capture Video,
- Send Video Message.
- Click on Download/Install codec button
- The following messages should appear
- Connect to FTP host: xx.xx.xx.xxx (This should be the IP address of the server)
- Connected
- Log in server
- Logged in
- Start downloading Codec file
- Downloading completed
- The Video Codec was successfully installed. Disconnect
- To verify that the codec was downloaded successfully, go to the My Documents directory on the MDA. A file called DtvMDADemo.exe should be present with the date and timestamp of when it was downloaded.
- 4.2 Recording a Video Message with the Handset
- In this version of the demo, the MDA-II handset is used to encode/compress high quality uncompressed live video input from an external video camcorder. While the MDA-II has a VGA capture camera, it can only capture still images at that resolution. The video capture of the MDA-II is limited to QCIF, 10 fps video that is automatically compressed in 3GPP format on the device.
- 4.2.1 Connecting the Camcorder to the PC
-
- Connect the Canon camcorder to the PC using the Firewire (1394) cable. The camcorder has a 4-pin connector receptacle.
- Once the Firewire cable is connected and the camcorder is turned on in “Camera” mode, the PC OS (Windows XP Professional in this demo version) will prompt for capture with several options.
- Select “Windows Movie Maker” which will bring in the Movie Maker application.
- 4.2.2 Capture a Video Sequence from the Camcorder on the PC
-
- Within Movie Maker select “Capture from video device” and choose a name for the captured file and location to store it on the PC.
- The next screen will ask for video settings; select “Digital Device Format (DV-AVI)”. That will capture the video in DV format, which is a high quality (almost lossless) video capture format.
- The next screen will show a preview window of the captured video as well as interface to “Start Capture” and “Stop Capture”. Since many wireless handsets do not have a large amount of onboard memory, it is suggested that 2-3 seconds of full motion video (60-90 frames) is recorded for faster computation.
- Once the video sequence is recorded, the Windows Movie Maker program can be exited.
- For this document it is assumed that the name of the recorder file is “testDV.avi” (note that the file is in AVI format—interleaved audio-video file with all individual frames arranged one after the other).
- 4.2.3 Uncompress the DV Video Sequence on the PC
-
- The recorded video in the previous step is in compressed DV format (720×480 pixels, 30 fps at about 28 Mbps) with integrated audio. To simulate full motion capture, the video should be uncompressed in common UYVY format, scaled down to VGA size (640×480 pixels), and the audio information removed.
- For this demo, it is necessary to create two source files for the MDA to encode: one for VGA (640×480) @ 30 fps and one for QCIF (160×144) @ 15 fps.
- From the VirtualDub1.6.3 directory run the “VirtualDub.exe” executable.
- a. In “File” use the “Open video file” option to load the “testDV.avi” file
- b. In “Audio” select “No audio” option
- c. In “Video” select “Filters” option. Choose “Add” and scroll down the menu to locate the “resize” filter. Set the “New width” at 640 and “New height” at 480. Set “Filter Mode” at “Precise Bicubic (A=0.75)”.
- d. In “Video” select “Color depth”. Set in the right column the output format to “4:2:2 YCbCr (UYVY)” format
- e. Make sure that in “Video” the “Compression” option is set to “Uncompressed RGB/YCbCr”.
- f. If a very long video sequence has been captured, it can be shortened by selecting in “Video” the “Select range” option. Typically 60-90 frames (2-3 seconds of video) is a reasonably large size of video to manipulate. The start can be from the beginning (“Start offset” of 0) or somewhere in the middle of the sequence, depending on the video that is captured. For example a 2 seconds (60 frames) uncompressed UYVY video VGA (640×480) sequence is 36 MB of data on the computer.
- g. Select “File” and “Save as AVI . . . ” to save the resulting uncompressed file. Call the file “testVGA_UYVY.avi”. This is now the raw (uncompressed) UYVY VGA input file to be encoded/compressed on the MDA-II handset.
- To create a file for QCIF (recommended for sending a video clip to another handset via a GPRS connection):
- In step c, enter 160 for the “New width”. Enter 144 for the “New height”
- In step g, name the file “testQcif_UYVY.avi”.
- 4.2.4 Transfer the Uncompressed Video Sequence to the MDA-II Handset
-
- Transfer the uncompressed video files “testVGA_UYVY.avi” and “testQcif_UYVY.avi” from the PC to the MDA
- Copy these files into the My Documents directory on the MDA-II for compression.
- For intermediate storage, the above large source files (uncompressed video input) can also be placed on the “Storage Card” of the device, and then copied to the My Documents directory for compression.
- Disconnect the MDA from the PC
- At this point the uncompressed video sequence is in the handset and ready to be compressed with the present invention's “Droplet” encoder software previously downloaded and installed on the handset
- 4.2.5 Compressing the Native Video Sequence in the MDA-II Handset
- Now that the video codec and the uncompressed video sequence have been successfully downloaded to the MDA-II handset, the handset is ready to perform the encode/compression.
-
- On the MDA, go to the MyDocuments directory
- Find and select the “DtvMDADemo” application.
- A UI window will pop up allowing the user to enter information:
- Source file: The name of the file containing the uncompressed video in UYVY format. (This was the file that was generated in the previous step)
- Destination file: The name of the file where the compressed video sequence will be stored (The compressed file will have an extension ending in .dtv). For this demo, leave the file name at bitstream.dtv
- Horizontal and vertical frame size.
- For encoding VGA video, the parameters will be 640 and 480 respectively.
- For encoding QCIF video, the parameters will be 160 and 144 respectively.
- Type of input file: YUV 4:2:2 by default
- Compression rate: Level 12 by default
- Range of frames to be compressed: Typically set to “All”. (Note: if the user chooses to change this, the total number of frames specified must be an even number because DTV processes 2 frames as a group of pictures)
- Select “Encode”, “Decode”, or both.
- Select Start.
- Upon successful completion, the compressed bitstream.dtv file specified in the destination file field above will have been created in the directory in “My Documents” on the MDA-II.
- If the file name was changed, it is important to keep the .dtv file extension.
- Reminder: the above encoding should be done twice, once for VGA/30 fps and once for QCIF/15 fps
- The decompressed QCIF files are also kept in “My Documents\DTV Output” on the MDA-II, for MMS transmission to other handsets via the GPRS network.
- 4.3 Transfer/Playback of VGA/30 fps Video Message on PC
- To play the encoded VGA file on the PC requires transferring the file from the MDA-II to the PC. At current GPRS data transfer rates (˜20-40 Kb/s), it should take approximately 16 seconds to send 1 second of compressed VGA video over the mobile network.
Higher speed 3 G or WiFi networks will eventually enable more efficient transfer of full VGA/30 fps video files. For this demo, the USB connection between the MDA-II handset and the PC is used to expedite the file transfer. -
- Connect the MDA-II to the PC via the cradle/USB. (Note that this is the standard method by which current digital cameras & camcorders are connected to home PCs in order to transfer digital photos and videos)
- Copy the encoded/compressed VGA file (\My Document\bitstream.dtv) from the MDA-II to the PC. The file should be copied to the following directory: “C:\DTV_PCplayer_Demo\”.
- Since the MDA-II device is equipped with a USB1 port, faster transfer of the VGA/30 fps files to a USB2 enabled PC can be accomplished by first copying them to the removable memory card in the MDA-II, and then removing the card and using a USB2 card reader for transfer, by connecting it directly to the USB2 port on the PC.
- Once file transfer is complete, go to the “C:\DTV_PCplayer\_Demo” directory in the PC and double click the file “dtvplayer.grf” or “dtvplayer_Win2K.grf” (OS dependent), and then click on the “Play” button (or select “Play” menu item) to view the VGA/30 fps video.
- To stop a clip you can simply exit the application (GraphEdit).
- 4.4 Sending QCIF/15 fps Video as an MMS Message via GPRS
- This section will demonstrate the ability to send the compressed QCIF/15 fps video from the MDA-II handset to the MMS server via GPRS. From there, an SMS notification will be sent to the targeted handset (in this case the Xphone), indicating that a video MMS is ready for download and playback. Alternatively, an email notification will be sent if the targeted receiving device is a networked computer.
- 4.4.1 Sending QCIF/15 fps Video to MMS Server via GPRS
-
- Click on the MDA-II Start menu and a Droplet icon will appear on the menu list.
- Select the Droplet icon and a new window pops up. It will have 3 button selections:
- Download/Install codec
- Capture Video,
- Send Video Message.
- Click on the “Send Video Message” button
- A new window to select files will open.
- Select the file that you want to send (in this case the compressed QCIF/15 fps video file).
- A new window will open, to enter the target phone number/email address.
- If an email address is entered (determined by the presence of the @ symbol) then the selected file will be sent with email notification.
- If a string of digits without “@” is entered, then the file/SMS notification will be sent to mailto:“string”@tmomail.net, corresponding in this case to a T-Mobile subscriber with the phone number entered as “string”.
- To send the compressed video as an MMS message to the Xphone, enter the handset's phone number.
- The user will be prompted to establish the GPRS connection. This can be accomplished, for example, by launching Internet Explorer and going to any well-known URL.
- Once the GPRS connection has been established successfully, click “OK”
- A new window opens where the following status messages should appear.
- Connect to FTP host: xx.xx.xx.xxx (This should be the IP address of the MMS server)
- Connected
- Log in server
- Logged in
- Changing directory to: public_html
- Changed
- Start uploading video file: <selected video file name>.dtv
- Uploading completed
- Start uploading link file: <selected video file name>.lnk
- Uploading completed
- The video file was successfully sent. Disconnect.
- Note: Two new files will show up in the public_html directory on the ftp server (<filename>.dtv and <filename>.lnk)
- 4.4.2 SMS Notification is Sent from the MMS Server
-
- A script running on the server polls the ftp location public_html and determines that new files are present.
- When a new file is present, the server script will parse the <new file>.lnk file and extract the name of the video file to be sent and the destination handset # or email address.
- The script will then send the SMS notification message to the target destination either via email or mobile SMS.
- 4.4.3 Receiving Handset Downloads Video Message and DTV Decoder
- This section will demonstrate the ability to receive the SMS notification on the Xphone, and to connect to the MMS server and download the QCIF/15 fps video file together with the DTV decoder. Upon receipt of the video file and decoder, the file will be decoded and played on the Xphone.
-
- The Xphone will receive an SMS message.
- Open the SMS message. Inside will be the location from which the video file can be downloaded.
- Downloading the DTV video file.
- Open Internet Explorer
- Type in the URL under which the video file is located
- The video file will be downloaded to the Xphone.
- Additionally, the Xphone video decoder needs to be downloaded (if it is not already present on the Xphone). Once downloaded, the Xphone decoder file (DtvXphoneDemo.exe) should be placed into the My Documents directory on the Xphone.
- Run the DtvXphoneDemo.exe
- A UI window will pop up allowing the user to enter information (the application will default to processing the bitstream.dtv file):
- Horizontal and vertical frame size.
- For decoding QCIF video, the parameters will be 160 and 144 respectively.
- Range of frames to be compressed: Typically set to “All”. (Note: if the user chooses to change this, the total number of frames specified must be an even number because DTV processes 2 frames as a group of pictures)
- All other fields will be ignored.
- Select “Decode”.
- Select Start.
- This will create an uncompressed AVI file, which can be played by most video players found on mobile handsets.
- To play the video file on the Xphone, simply click on the created file and the resident Microsoft Media Player will play the video.
- 4.4.4 Use of Alternate FTP Connection on Xphone
- In order to demonstrate the flexibility enabled by the present invention's Droplet all-software video messaging platform, the use of a simple Smartphone FTP application to download the QCIF/15 fps video messages and Droplet decoder to the Xphone handset is also demonstrated.
-
- The Orneta FTP application for Windows Smartphones devices on the Xphone is used.
- The Orneta FTP application installer can be downloaded to the PC from: http://www.handango.com/PlatformProductDetail.jsp?productType=2&platformId=11&siteId=1&SectionId=0&catalog=1&productId=87548
- Follow the instructions to install Orneta FTP using the PC with the Xphone connected
- Obtain a free Registration Code from: http://x.msmobiles.com/free-smartphone-software/default.aspx
- To use the Orneta FTP application on the Xphone to download video messages and the Droplet decoder from the MMS server:
- Start a GPRS connection
- Launch the Orneta FTP application from the Start menu
- Under Menu/Settings/Set Download Folder, select choose \windows
- Connect to the specified MMS server (same as in Section 4.1, for example)
- Download: select the Droplet video encoder “DtvXphoneDemo.exe”
- Under Menu/Settings/Set Download Folder, select \Storage\My Documents\
- Connect to the MMS server again
- Download: select the QCIF/15 video file, i.e. “bitstream.dtv”
- Exit
This concludes the discussion of Example 1.
- Aspects of the present invention comprise, in part, an all-software camcorder phone application capable of real-time capture of full (VGA)-size images (640×480) at 30 frames per second (fps), which may use only a single standard RISC processor already incorporated in the vast majority of multimedia handsets. In contrast, within the battery power constraints of mobile handsets, current MPEG-based camcorder phones support real-time capture of images that are limited to QCIF or CIF size ( 1/16th or ¼ the size of VGA) at 4-15 fps. Yet even these small, choppy video clips require complex and expensive handset platform designs, in which the video functions are implemented as a combination of hardware and software, and partitioned between multiple processing devices: RISC processors, ASIC, and DSPs.
- With regard to the mobile operator environment, aspects of the present invention's low-complexity video processing and distribution technologies are integrated into a powerful new and inventive all-software video messaging platform that enables turnkey deployment using existing mobile handsets and mobile Multimedia Messaging Service controller (MMSC) infrastructure. Complementing the above mobile camcorder application, embodiments of the present invention's content management platform provide modules for integrating the invention's compressed images and videos together with sounds and text into complete mobile multimedia messages and “ring-tones”, along with on-the-fly editing, thumbnail previews, multimedia mailboxes, on-line repository services, and subscription management.
- Aspects of the present invention's video codecs offer customers a 30-40× reduction in power consumption (both SW and HW implementations—see Table 1) when compared to optimized MPEG-2/MPEG-4 codecs. HW product implementation costs are significantly reduced via a 10× reduction in the number of CMOS gates required, from approximate ˜1 million to ˜100,000, and hence in the corresponding silicon real estate requirements. For full size (VGA) and full-frame-rate (30 fps) video processing, the present invention's innovative video codec designs also reduce internal memory requirements from several megabytes to 128 kilobytes, freeing up on-board memory resources in mobile handsets for other revenue-generating features and applications. The present invention's codecs are also able to efficiently process both still images and video, and can thus replace separate MPEG and JPEG codecs with a single lower-cost and lower-power solution.
- Although significantly applicable at supporting the deployment of VGA, 30 fps camcorder phones and related services, the present invention's unique mobile video platform technologies also offer significant benefits across a broad range of other mobile video services, via a combination of: scalable image size: QCIF (176×144)-D1 (720×480), simplified video editing (cuts, inserts, text overlays, etc.), simplified synchronization with voice codecs, and low latency for enhanced video streaming performance.
TABLE 1 Codec Performance Comparison: Mobile Handset Applications Hardware Codecs (ASIC) Power Consumption Company Image Format Frame Rate (mW) Hantroref 1 QCIF 15 fps 11 Fujitsuref 2 QCIF 15 fps 13 Droplet VGA 30 fps 10 SW Codecs Processor Utilization Company Image Format Frame Rate (MHz on ARM-9) Hantro/Motorolaref 3 QCIF 15 fps 190 Droplet QCIF 15 fps 30 Droplet VGA 30 fps 150 - The present invention also comprises MMS infrastructure products enabling deployment of premium video messaging services in conjunction with the inventive SW video codec/camcorder applications for mobile handsets. Additional aspects of the invention comprise advanced transcoding applications support complete interoperability with other commonly-deployed standards-based and proprietary video formats. Additionally included is a content management platform that provides modules for integrating the invention's compressed images and videos together with sounds and text into complete mobile multimedia messages and “ring-tones”, along with a suite of corresponding MMS message management capabilities. This content management platform can be used by wireless operators and MMS service providers both as a set of SW modules, for rapid and cost-effective upgrades to existing MMS infrastructure, and as a stand-alone server for new MMS controller installations. The inventive MMS infrastructure products may include:
- Product Description
-
- DTV-VGT Video Gateway Transcoder: SW Transcoder application for upgrading existing MMS Video Gateways to support conversion of video content between Droplet DTV format and other video formats such as MPEG-2, MPEG-4, Motion-JPEG, Microsoft Media, and RealVideo.
- DTV-CMP SW Content Management Platform: Suite of content management SW modules for upgrading existing MMS Message Application Servers: creation of MMS messages and “ring-tones” that integrate the present invention's compressed images and videos together with sounds and text, on-the-fly editing, thumbnail previews, multimedia mailboxes, on-line repository services, and subscription management.
- DTV-CMS Content Management Server: Server-based integrated SW content management platform, for new MMSC deployments.
- The present invention also comprises a Content Management Service Platform that with SW Modules or Stand-Alone Server may include:
-
- Mobile Multimedia Composer: Integrates the present invention's improved wavelet-compressed images and videos with sounds and text in one message.
- Preview Player: Provides “thumb-nail” previews of the present invention's wavelet-compressed images, videos, and integrated MMS messages
- Mobile Multimedia Editor: Enables on-the-fly editing of the present invention's wavelet-compressed images, videos, and integrated MMS messages with tools and filters.
- Multimedia Ring-Tone Creator: Allows users to create personal multimedia “ring-tones”, by combining polyphonic ring-tones and other sounds with wavelet-compressed images and videos.
- Mobile Multimedia Album or “Mblog”: Repository for the present invention's wavelet-compressed images, videos, and integrated MMS messages.
- Mobile Multimedia Subscription Management: Copy/forward the present invention's wavelet-compressed images, videos, and integrated MMS messages; purchase additional storage; purchase DVD hard copies.
- Mobile Multimedia Mail-Box: SMS-managed In-box and Out-box for the present invention's integrated MMS messages.
- Mobile Multimedia Address Book: Management of Mobile Multimedia contacts.
- It should be noted that embodiments of the present invention may provide:
-
- Flexible, rapid, and cost-effective OTA/OTN upgrades increase ROI from installed MMS infrastructure
- Advanced transcoding supports complete interoperability with other commonly-deployed standards-based and proprietary video formats.
- Content management platform available as suite of SW modules for upgrading existing MMS infrastructure, or as a stand-alone server for new MMS installations
- The present invention's DTV-JVC Java Video Codec generates decompressed video images that support all Player Functionality defined in Java Community Process JSR-135 including the following:
Int getDisplayHeight( ) Return the actual height of the current render video. Int getDisplayWidth( ) Return the actual width of the current render video. Int getDisplayX( ) Return the X-coordinate of the video with respect to the GUI object where the video is displayed. Int getDisplayY( ) Return the Y-coordinate of the video with respective to the GUI object where the video is displayed. byte[ ] getSnapshot(java.lang.String imageType) Get a snapshot of the displayed content. int getSourceHeight( ) Return the height of the source video. int getSourceWidth( ) Return the width of the source video. java.lang.Object initDisplayMode(int mode, java.lang.Object arg) Initialize the mode on how the video is displayed. Void setDisplayFullScreen(boolean fullScreenMode) Set the size of the render region for the video clip to be fullscreen. Void setDisplayLocation(int x, int y) Set the location of the video with respect to the canvas where the video is displayed. void setDisplaySize(int width, int height) Resize the video image. Void setVisible(boolean visible) Show or hide the video. - The present invention's DTV-JVC Java Video Codec generates decompressed video images that support all Player Effect Controls defined in Java Community Process JSR-234 including the following:
- mageFilterControl
- ImageFilterControl is an image effect that can be used to set various image filters such as monochrome and negative.
- ImageTonalityControl
- ImageTonalityControl is an effect that can be used to set various image settings such as brightness, contrast, and gamma.
- ImageTransformControl
- ImageTransformControl is used to crop, zoom, mirror, flip, stretch, and rotate images.
- OverlayControl
- OverlayControl controls the setting of overlay images on top of video or still images.
- WhiteBalanceControl
- WhiteBalanceControl is an image/video effect for altering the white balance.
- The present invention also comprises products, methods and processes for establishing, providing and operating a mobile video blog service. This service provides every user having a video phone with the ability to: shoot, edit, save, share, and “publish” their personal videos and movies online.
- With respect to users, aspects of the present invention's products for mobile video blog service, code named “Mobedia”, provide:
-
- 1. The Mobedia SW Camcorder application which can be pre-installed in mobile handsets, or users can download and install it on any Java enabled video phone.
- 2. Using the Mobedia SW Camcorder application, users can record full VGA/30 fps (or even higher quality) video using their mobile handsets.
- 3. Using the Mobedia SW Cinema application, users can browse/edit takes, add titles, etc., and “splice” multiple takes to create personal movies, using the present invention's SW camcorder application.
- 4. A simple version of the Mobedia SW Cinema application may be distributed to users free of charge, while a more powerful “Cinema-Pro” version may be purchased by users.
With regard to servers, termed Mobedia servers, aspects of the present invention may provide: - 5. Using Mobedia handset client SW, users can send Video takes to Mobedia servers, via mobile, fixed wireless, or wireline connections.
- 6. Mobedia subscription service allows users to archive video takes and movies on the server (paid storage), carry out further editing online, download and save video in a variety of popular formats, or order copies on DVD (paid).
- 7. Mobedia subscription service allows users to create movie albums on the Mobedia site, and invite friends, family, colleagues, etc. to view their movies, or order their own copies on DVDs (paid) as gifts, etc.
With respect to friends, family, and etc., Mobedia type aspects of the present invention provide: - 8. Friends, family, colleagues, etc., receive email invitations to view the user's movies, download and save copies (paid), or order their own copies on DVD (paid).
With respect to general audience (cinema model), Mobedia aspects of the present invention provide: - 9. Droplet's Mobedia subscription service allows users to “publish” their movies for the General Public to view on the Mobedia Cinema site.
- 10. The General Audience can search the archive of published movies by ranking, subject, category, etc., and view free previews on the Mobedia Cinema site.
- 11. The General Audience can pay to view, download, or order a copy on DVD.
- The following methods and processes comprise aspects of the present invention and are exclusively enabled by the present technology.
- Revenue Sources/Business Model:
-
-
- Film makers: pay for server space, downloading files, and ordering DVDs. Pay for enhanced versions of editing software.
- Friends, family, etc.: pay for downloading files and ordering DVDs
- General Audience: pays for watching full movies (only previews are free), ordering DVDs or downloads.
- Film maker may get a share of fees paid by General Audience
- Mobile Operators: pay for increased data traffic via revenue-sharing model
Service components of aspects of the present invention comprise: - the present invention's Mobile Video Messaging Platform (server and handset SW applications) to capture, send, and manage personal video content
- the present invention's Mobedia SW Video Camcorder application enables video shooting on any java/video phone
- the present invention's Mobedia SW Video Cinema client applications include basic video production, editing, and viewing technology (simple and Pro versions)
- the present invention's Mobedia SW Video Cinema web-based Content Management applications support Mobedia Cinema movie albums, personal moving sharing, and Mobedia Cinema “publishing”.
- Presented herein is an improved mobile imaging application, handset architecture, and service platform architecture, which combine to substantially reduce the technical complexity and costs related with offering high-quality still and video imaging services to mobile subscribers. Improved adaptive joint-source channel coding technique is the corresponding ability of wireless carriers and MMS service providers to offer a greater range of quality-of-service (QoS) performance and pricing levels to their consumer and enterprise customers, thus maximizing the revenues generated using their wireless network infrastructure. Improved adaptive joint-source channel coding, based on algorithms with higher computational efficiency, enables support for a much higher level of network homogeneity, in terms of channel types (wireless and wire line), channel bandwidths, channel noise/error characteristics, user devices, and user services. Additionally provided are methods, devices, processes and business methods providing innovative and enhanced services in the field of still and moving video in the mobile telephone fields.
- Additionally provided as aspects of the present invention are the following in summary form.
- Mobile imaging applications utilizing improved wavelet-based codec, as an all-SW implementation, an all-HW implementation, or as a hybrid SW+HW implementation.
- Also provided are systems and methods comprising improved joint source-channel coding using fine grain scalability of the improved wavelet-based codec described above, utilizing information on both instantaneous and predicted channel bandwidth and error conditions in all three of the source coder, the channel coder, and the adaptive rate controller to maximize control of both the instantaneous and average quality (video rate vs. distortion) of the reconstructed video signal. Also provided are systems and methods that can be applied to a greater range of quality-of-service (QoS) performance and pricing levels for consumer and enterprise MMS customers, support for a much higher level of network heterogeneity, in terms of channel types (fixed wireless, mobile wireless, and wire line), channel bandwidths, channel noise/error characteristics, user devices, and user services, including improved multicasting.
- Also provided is a mobile camcorder application—combining aspects of the two preceding paragraphs with related image pre-processing and post-processing functions, and voice recording, for full camcorder capability in mobile devices, either as an all-SW implementation, an all-HW implementation, or as a hybrid SW+HW implementation.
- Also provided is a mobile imaging application utilizing improved wavelet-based codec, implemented as Java application, as an all-SW implementation, an all-HW implementation, or as a hybrid SW+HW implementation.
- Also provided is a mobile camcorder application—combining the application of the preceding paragraph above with related image pre-processing and post-processing functions, and voice recording, for full camcorder capability in mobile devices, either as an all-SW implementation, an all-HW implementation, or as a hybrid SW+HW implementation.
- Also provided is imaging-enabled mobile handset architecture using as pects and features of the preceding paragraphs of this summary, where the mobile imaging application is incorporated in the handset baseband multimedia processing section of the handset, in the imager module, or in a removable storage medium.
- Also provided is OTA delivery or upgrade of the above features of this summary above into imaging-enabled handset described above.
- Also provided is a system enabling point of sale installation or upgrade of the above features and systems above into an imaging-enabled handset.
- Also provided is a mobile imaging transcoder for universal compatibility of the above features of this summary with other standards-based or proprietary imaging formats—all SW application delivered to and installed in an MMSC Video Gateway via automated OTN upgrade or via manual procedures.
- Also provided is a mobile imaging service platform architecture, method and system—combining all the features of this summary.
- While the above is a description of certain aspects of embodiments of the invention, various alternatives, modifications, and equivalents may be used. Therefore, the above description should not be taken as limiting the scope of the invention which is defined by the appended claims.
Claims (2)
1. A method of deploying an imaging service platform comprising the steps of:
providing a transcoder application on a download server connected to a network;
signaling the network that the transcoder application is available for deployment; and
deploying the transcoder application over the network from the download server to a video gateway located on the network.
2. The method of claim 1 , further comprising the step of automatically installing the deployed transcoder application on the video gateway.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/357,661 US20060218482A1 (en) | 2002-04-19 | 2006-02-16 | Mobile imaging application, device architecture, service platform architecture and services |
US13/037,296 US8849964B2 (en) | 2002-04-19 | 2011-02-28 | Mobile imaging application, device architecture, service platform architecture and services |
US14/339,625 US20140369671A1 (en) | 2002-04-19 | 2014-07-24 | Mobile imaging application, device architecture, service platform architecture and services |
Applications Claiming Priority (28)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US37406902P | 2002-04-19 | 2002-04-19 | |
US37397402P | 2002-04-19 | 2002-04-19 | |
US37396602P | 2002-04-19 | 2002-04-19 | |
US37406102P | 2002-04-19 | 2002-04-19 | |
US38525002P | 2002-05-28 | 2002-05-28 | |
US38525302P | 2002-05-28 | 2002-05-28 | |
US38525402P | 2002-05-28 | 2002-05-28 | |
US39038002P | 2002-06-21 | 2002-06-21 | |
US39049202P | 2002-06-21 | 2002-06-21 | |
US39034502P | 2002-06-21 | 2002-06-21 | |
US39038302P | 2002-06-21 | 2002-06-21 | |
US10/418,831 US6825780B2 (en) | 2002-04-19 | 2003-04-17 | Multiple codec-imager system and method |
US10/418,363 US20030198395A1 (en) | 2002-04-19 | 2003-04-17 | Wavelet transform system, method and computer program product |
US10/418,649 US20030206597A1 (en) | 2002-04-19 | 2003-04-17 | System, method and computer program product for image and video transcoding |
US10/447,455 US20030229773A1 (en) | 2002-05-28 | 2003-05-28 | Pile processing system and method for parallel processors |
US10/447,514 US7844122B2 (en) | 2002-06-21 | 2003-05-28 | Chroma temporal rate reduction and high-quality pause system and method |
US50714703P | 2003-09-30 | 2003-09-30 | |
US50714803P | 2003-09-30 | 2003-09-30 | |
US10/944,437 US20050104752A1 (en) | 2002-04-19 | 2004-09-16 | Multiple codec-imager system and method |
US61231104P | 2004-09-21 | 2004-09-21 | |
US61265204P | 2004-09-22 | 2004-09-22 | |
US61265104P | 2004-09-22 | 2004-09-22 | |
US10/955,240 US20050105609A1 (en) | 2003-09-30 | 2004-09-29 | System and method for temporal out-of-order compression and multi-source compression rate control |
US65405805P | 2005-02-16 | 2005-02-16 | |
US11/232,165 US7525463B2 (en) | 2003-04-17 | 2005-09-20 | Compression rate control system and method with variable subband processing |
US11/232,725 US20060072834A1 (en) | 2003-04-17 | 2005-09-21 | Permutation procrastination |
US11/232,726 US7436329B2 (en) | 2003-04-17 | 2005-09-21 | Multiple technique entropy coding system and method |
US11/357,661 US20060218482A1 (en) | 2002-04-19 | 2006-02-16 | Mobile imaging application, device architecture, service platform architecture and services |
Related Parent Applications (9)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/418,649 Continuation-In-Part US20030206597A1 (en) | 2002-04-19 | 2003-04-17 | System, method and computer program product for image and video transcoding |
US10/418,363 Continuation-In-Part US20030198395A1 (en) | 2002-04-19 | 2003-04-17 | Wavelet transform system, method and computer program product |
US10/447,455 Continuation-In-Part US20030229773A1 (en) | 2002-04-19 | 2003-05-28 | Pile processing system and method for parallel processors |
US10/447,514 Continuation-In-Part US7844122B2 (en) | 2002-04-19 | 2003-05-28 | Chroma temporal rate reduction and high-quality pause system and method |
US10/944,437 Continuation-In-Part US20050104752A1 (en) | 2002-04-19 | 2004-09-16 | Multiple codec-imager system and method |
US10/955,240 Continuation-In-Part US20050105609A1 (en) | 2002-04-19 | 2004-09-29 | System and method for temporal out-of-order compression and multi-source compression rate control |
US11/232,165 Continuation-In-Part US7525463B2 (en) | 2002-04-19 | 2005-09-20 | Compression rate control system and method with variable subband processing |
US11/232,725 Continuation-In-Part US20060072834A1 (en) | 2002-04-19 | 2005-09-21 | Permutation procrastination |
US11/232,726 Continuation-In-Part US7436329B2 (en) | 2002-04-19 | 2005-09-21 | Multiple technique entropy coding system and method |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/037,296 Continuation US8849964B2 (en) | 2002-04-19 | 2011-02-28 | Mobile imaging application, device architecture, service platform architecture and services |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060218482A1 true US20060218482A1 (en) | 2006-09-28 |
Family
ID=46323847
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/357,661 Abandoned US20060218482A1 (en) | 2002-04-19 | 2006-02-16 | Mobile imaging application, device architecture, service platform architecture and services |
US13/037,296 Expired - Fee Related US8849964B2 (en) | 2002-04-19 | 2011-02-28 | Mobile imaging application, device architecture, service platform architecture and services |
US14/339,625 Abandoned US20140369671A1 (en) | 2002-04-19 | 2014-07-24 | Mobile imaging application, device architecture, service platform architecture and services |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/037,296 Expired - Fee Related US8849964B2 (en) | 2002-04-19 | 2011-02-28 | Mobile imaging application, device architecture, service platform architecture and services |
US14/339,625 Abandoned US20140369671A1 (en) | 2002-04-19 | 2014-07-24 | Mobile imaging application, device architecture, service platform architecture and services |
Country Status (1)
Country | Link |
---|---|
US (3) | US20060218482A1 (en) |
Cited By (75)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050125733A1 (en) * | 2003-12-05 | 2005-06-09 | Ati Technologies, Inc. | Method and apparatus for multimedia display in a mobile device |
US20050226170A1 (en) * | 2004-04-07 | 2005-10-13 | Sandeep Relan | System, method, and apparatus for secure sharing of multimedia content across several electronic devices |
US20060085534A1 (en) * | 2002-04-19 | 2006-04-20 | Ralston John D | Video monitoring application, device architectures, and system architecture |
US20060170232A1 (en) * | 2005-01-25 | 2006-08-03 | Hammond Virgil E Iii | CINECOACH production systems |
US20060229029A1 (en) * | 2005-04-07 | 2006-10-12 | Intel Corporation | Ultra high frequency / very high frequency (UHF/VHF) band enhancement |
US20070288970A1 (en) * | 2006-05-11 | 2007-12-13 | Mats Tedenvall | Method and apparatus for sharing tv content with a remote device |
US20080059648A1 (en) * | 2006-09-01 | 2008-03-06 | Freedom Broadcast Network, Llc | Dynamically Configurable Processing System |
US20080059618A1 (en) * | 2006-08-30 | 2008-03-06 | Research In Motion Limited, | Automatic attachment of image and/or audio records to electronic calendar meeting event record in portable wireless devices |
US20080071895A1 (en) * | 2006-09-19 | 2008-03-20 | Bellsouth Intellectual Property Corporation | Imaging viewer systems and methods |
US20080182558A1 (en) * | 2007-01-31 | 2008-07-31 | Samsung Electronics Co., Ltd. | Contents management method and apparatus |
US20080195698A1 (en) * | 2007-02-09 | 2008-08-14 | Novarra, Inc. | Method and System for Transforming and Delivering Video File Content for Mobile Devices |
US20080198264A1 (en) * | 2007-02-16 | 2008-08-21 | Nikhil Balram | Methods and systems for improving low resolution and low frame rate video |
US20080256169A1 (en) * | 2007-04-12 | 2008-10-16 | Hubert Oehm | Graphics for limited resolution display devices |
US20080260031A1 (en) * | 2007-04-17 | 2008-10-23 | Qualcomm Incorporated | Pixel-by-pixel weighting for intra-frame coding |
US20080313686A1 (en) * | 2007-06-13 | 2008-12-18 | Matvey Thomas R | Handheld camcorder accessory with pre-programmed wireless internet access for simplified webcasting and handheld camcorder with built-in pre-programmed wireless internet access for simplified webcasting and method of commercially supplying and supporting same |
US20090034606A1 (en) * | 2007-07-30 | 2009-02-05 | Macinnis Alexander G | Display device with conversion capability for portable media player |
US20090150350A1 (en) * | 2007-12-05 | 2009-06-11 | O2Micro, Inc. | Systems and methods of vehicle entertainment |
US20090171984A1 (en) * | 2004-10-30 | 2009-07-02 | Deuk Hee Park | Apparatus and method for interlinking the classified objects having the property and computer readable medium processing the method |
US20090177476A1 (en) * | 2007-12-21 | 2009-07-09 | May Darrell | Method, system and mobile device for registering voice data with calendar events |
US20100161635A1 (en) * | 2008-12-22 | 2010-06-24 | Spondon Dey | Method and apparatus for providing a mobile video blog service |
US20100191780A1 (en) * | 2009-01-29 | 2010-07-29 | Colin Kroll | Uploading a Hierarchical File Structure Using a Web Service |
US20100210240A1 (en) * | 2009-02-17 | 2010-08-19 | Flexilis, Inc. | System and method for remotely securing or recovering a mobile device |
US20100265347A1 (en) * | 2009-04-16 | 2010-10-21 | Sony Ericsson Mobile Communications Ab | Methods, electronic devices, and computer program products for encoding image files to facilitate browsing thereof |
US20100281042A1 (en) * | 2007-02-09 | 2010-11-04 | Novarra, Inc. | Method and System for Transforming and Delivering Video File Content for Mobile Devices |
US20100325629A1 (en) * | 2009-03-24 | 2010-12-23 | Robb Fujioka | Apparatus, System and Method for a User Profiled-Based OS for Mobile Devices |
US20110047033A1 (en) * | 2009-02-17 | 2011-02-24 | Lookout, Inc. | System and method for mobile device replacement |
US20110047597A1 (en) * | 2008-10-21 | 2011-02-24 | Lookout, Inc., A California Corporation | System and method for security data collection and analysis |
US20110047620A1 (en) * | 2008-10-21 | 2011-02-24 | Lookout, Inc., A California Corporation | System and method for server-coupled malware prevention |
US20110047594A1 (en) * | 2008-10-21 | 2011-02-24 | Lookout, Inc., A California Corporation | System and method for mobile communication device application advisement |
US20110119765A1 (en) * | 2009-11-18 | 2011-05-19 | Flexilis, Inc. | System and method for identifying and assessing vulnerabilities on a mobile communication device |
US20110151880A1 (en) * | 2009-12-21 | 2011-06-23 | Electronics And Telecommunications Research Institute | Fixed mobile convergence terminal using downloadable wideband voice codec, method thereof and method of providing wideband voice codec in call control server |
US20110181686A1 (en) * | 2003-03-03 | 2011-07-28 | Apple Inc. | Flow control |
US8099472B2 (en) | 2008-10-21 | 2012-01-17 | Lookout, Inc. | System and method for a mobile cross-platform software system |
US8365252B2 (en) | 2008-10-21 | 2013-01-29 | Lookout, Inc. | Providing access levels to services based on mobile device security state |
US8381303B2 (en) | 2008-10-21 | 2013-02-19 | Kevin Patrick Mahaffey | System and method for attack and malware prevention |
US8505095B2 (en) | 2008-10-21 | 2013-08-06 | Lookout, Inc. | System and method for monitoring and analyzing multiple interfaces and multiple protocols |
US8510843B2 (en) | 2008-10-21 | 2013-08-13 | Lookout, Inc. | Security status and information display system |
US20130325952A1 (en) * | 2012-06-05 | 2013-12-05 | Cellco Partnership D/B/A Verizon Wireless | Sharing information |
US8655307B1 (en) | 2012-10-26 | 2014-02-18 | Lookout, Inc. | System and method for developing, updating, and using user device behavioral context models to modify user, device, and application state, settings and behavior for enhanced user security |
US20140079369A1 (en) * | 2011-06-23 | 2014-03-20 | Huawei Device Co., Ltd. | Video Recording Method for Mobile Terminal, and Related Apparatus and System |
US8712471B2 (en) | 2004-07-16 | 2014-04-29 | Virginia Innovation Sciences, Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US8738765B2 (en) | 2011-06-14 | 2014-05-27 | Lookout, Inc. | Mobile device DNS optimization |
US20140153906A1 (en) * | 2010-03-09 | 2014-06-05 | Yolanda Prieto | Video enabled digital devices for embedding user data in interactive applications |
US20140198233A1 (en) * | 2013-01-14 | 2014-07-17 | Samsung Electronics Co., Ltd | Method for compressing image data collected by camera and electronic device for supporting the method |
US8788881B2 (en) | 2011-08-17 | 2014-07-22 | Lookout, Inc. | System and method for mobile device push communications |
US8805358B2 (en) | 2004-07-16 | 2014-08-12 | Virginia Innovation Sciences, Inc. | Method and apparatus for multimedia communications with different user terminals |
US8855601B2 (en) | 2009-02-17 | 2014-10-07 | Lookout, Inc. | System and method for remotely-initiated audio communication |
US8855599B2 (en) | 2012-12-31 | 2014-10-07 | Lookout, Inc. | Method and apparatus for auxiliary communications with mobile communications device |
US20140368672A1 (en) * | 2002-04-19 | 2014-12-18 | Soryn Technologies Llc | Methods for Deploying Video Monitoring Applications and Services Across Heterogeneous Networks |
US20150007131A1 (en) * | 2011-09-26 | 2015-01-01 | Norman Ortiz | System and Method for mobile application development using mobile devices |
US8984628B2 (en) | 2008-10-21 | 2015-03-17 | Lookout, Inc. | System and method for adverse mobile application identification |
US9043919B2 (en) | 2008-10-21 | 2015-05-26 | Lookout, Inc. | Crawling multiple markets and correlating |
US9042876B2 (en) | 2009-02-17 | 2015-05-26 | Lookout, Inc. | System and method for uploading location information based on device movement |
US9208215B2 (en) | 2012-12-27 | 2015-12-08 | Lookout, Inc. | User classification based on data gathered from a computing device |
US9215074B2 (en) | 2012-06-05 | 2015-12-15 | Lookout, Inc. | Expressing intent to control behavior of application components |
US9235704B2 (en) | 2008-10-21 | 2016-01-12 | Lookout, Inc. | System and method for a scanning API |
US20160162700A1 (en) * | 2013-01-12 | 2016-06-09 | Pro Softnet Corporation | Method for sharing multiple data items using a single url |
US9374369B2 (en) | 2012-12-28 | 2016-06-21 | Lookout, Inc. | Multi-factor authentication and comprehensive login system for client-server networks |
US9424409B2 (en) | 2013-01-10 | 2016-08-23 | Lookout, Inc. | Method and system for protecting privacy and enhancing security on an electronic device |
US9589129B2 (en) | 2012-06-05 | 2017-03-07 | Lookout, Inc. | Determining source of side-loaded software |
US9642008B2 (en) | 2013-10-25 | 2017-05-02 | Lookout, Inc. | System and method for creating and assigning a policy for a mobile communications device based on personal data |
US9729918B2 (en) | 2004-07-16 | 2017-08-08 | Virginia Innovation Sciences, Inc. | Method and system for efficient communication |
US9753796B2 (en) | 2013-12-06 | 2017-09-05 | Lookout, Inc. | Distributed monitoring, evaluation, and response for multiple devices |
US9781148B2 (en) | 2008-10-21 | 2017-10-03 | Lookout, Inc. | Methods and systems for sharing risk responses between collections of mobile communications devices |
US20170293900A1 (en) * | 2008-01-09 | 2017-10-12 | T-Mobile Usa, Inc. | Mobile application monitoring system |
US9955352B2 (en) | 2009-02-17 | 2018-04-24 | Lookout, Inc. | Methods and systems for addressing mobile communications devices that are lost or stolen but not yet reported as such |
US10122747B2 (en) | 2013-12-06 | 2018-11-06 | Lookout, Inc. | Response generation after distributed monitoring and evaluation of multiple devices |
US10218697B2 (en) | 2017-06-09 | 2019-02-26 | Lookout, Inc. | Use of device risk evaluation to manage access to services |
US10306229B2 (en) | 2015-01-26 | 2019-05-28 | Qualcomm Incorporated | Enhanced multiple transforms for prediction residual |
US10405021B2 (en) * | 2007-07-12 | 2019-09-03 | At&T Intellectual Property I, L.P. | System for presenting media services |
US10540494B2 (en) | 2015-05-01 | 2020-01-21 | Lookout, Inc. | Determining source of side-loaded software using an administrator server |
US10554840B2 (en) * | 2018-02-07 | 2020-02-04 | Kyocera Document Solutions Inc. | Image forming apparatus and non-transitory computer readable recording medium |
US10623774B2 (en) | 2016-03-22 | 2020-04-14 | Qualcomm Incorporated | Constrained block-level optimization and signaling for video coding tools |
US10956372B2 (en) | 2017-08-23 | 2021-03-23 | Bank Of America Corporation | Image capturing and processing for legacy format integration |
US11323748B2 (en) | 2018-12-19 | 2022-05-03 | Qualcomm Incorporated | Tree-based transform unit (TU) partition for video coding |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20130039431A1 (en) * | 2011-08-12 | 2013-02-14 | Electronics And Telecommunications Research Institute | Power-scalable encoding/decoding apparatus and method |
US9215368B2 (en) | 2012-12-02 | 2015-12-15 | Bachir Babale | Virtual decals for precision alignment and stabilization of motion graphics on mobile video |
JP6435276B2 (en) | 2013-01-28 | 2018-12-05 | サンダーリング・マネージメント・リミテッドSanderling Management Limited | Dynamic promotion layout management and distribution rules |
JP6203515B2 (en) * | 2013-03-29 | 2017-09-27 | 株式会社メガチップス | Image processing device |
US20140355665A1 (en) * | 2013-05-31 | 2014-12-04 | Altera Corporation | Adaptive Video Reference Frame Compression with Control Elements |
CN108933948B (en) * | 2017-05-24 | 2022-03-25 | 腾讯科技(深圳)有限公司 | Video playing progress synchronization method and device |
US11206297B2 (en) | 2018-03-19 | 2021-12-21 | Livescale Technologies Inc. | Video streaming |
CN112135052B (en) * | 2020-09-24 | 2022-06-14 | 张晏铭 | Desktop cloud-based photographing system and method |
Citations (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6029196A (en) * | 1997-06-18 | 2000-02-22 | Netscape Communications Corporation | Automatic client configuration system |
US6121998A (en) * | 1992-02-19 | 2000-09-19 | 8×8, Inc. | Apparatus and method for videocommunicating having programmable architecture permitting data revisions |
US6124882A (en) * | 1992-02-19 | 2000-09-26 | 8×8, Inc. | Videocommunicating apparatus and method therefor |
US6167567A (en) * | 1998-05-05 | 2000-12-26 | 3Com Corporation | Technique for automatically updating software stored on a client computer in a networked client-server environment |
US20010030667A1 (en) * | 2000-04-10 | 2001-10-18 | Kelts Brett R. | Interactive display interface for information objects |
US20020012347A1 (en) * | 2000-02-03 | 2002-01-31 | Patrick Fitzpatrick | System and method for downloading code |
US20020022522A1 (en) * | 2000-08-09 | 2002-02-21 | Konami Corporation | Game item providing system, method, and computer data signal |
US6356543B2 (en) * | 1997-11-25 | 2002-03-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Controlling mobile phone system user views from the world-wide web |
US20020100036A1 (en) * | 2000-09-22 | 2002-07-25 | Patchlink.Com Corporation | Non-invasive automatic offsite patch fingerprinting and updating system and method |
US20020143792A1 (en) * | 2001-03-27 | 2002-10-03 | Sabin Belu | Systems and methods for creating self-extracting files |
US20020184643A1 (en) * | 1999-12-16 | 2002-12-05 | Laurent Fichet | Tansmission of a command to a receiver or to a decoder |
US20030084283A1 (en) * | 2001-09-04 | 2003-05-01 | Pixton Jeffrey Seth | Digital broadcast system |
US6754894B1 (en) * | 1999-12-03 | 2004-06-22 | Command Audio Corporation | Wireless software and configuration parameter modification for mobile electronic devices |
US20040133914A1 (en) * | 2003-01-03 | 2004-07-08 | Broadq, Llc | Digital media system and method therefor |
US20040148362A1 (en) * | 2001-11-02 | 2004-07-29 | Lee Friedman | Systems and methods for managing and aggregating media formats |
US20040166834A1 (en) * | 2001-07-12 | 2004-08-26 | Omar Salim H. | System and method for providing remote data access and transcoding for a mobile communication device |
US6847403B1 (en) * | 1997-11-05 | 2005-01-25 | Polycom, Inc. | Integrated portable videoconferencing unit |
US20050038892A1 (en) * | 2003-08-13 | 2005-02-17 | Sun Microsystems, Inc. | Method for wireless communication and apparatus for conducting the same |
US20050036034A1 (en) * | 2003-08-15 | 2005-02-17 | Rea David D. | Apparatus for communicating over a network images captured by a digital camera |
US6876379B1 (en) * | 1999-11-24 | 2005-04-05 | Orange Personal Communications Services Limited | Mobile communications |
US20060072837A1 (en) * | 2003-04-17 | 2006-04-06 | Ralston John D | Mobile imaging application, device architecture, and service platform architecture |
US7266611B2 (en) * | 2002-03-12 | 2007-09-04 | Dilithium Networks Pty Limited | Method and system for improved transcoding of information through a telecommunication network |
US20080072261A1 (en) * | 2006-06-16 | 2008-03-20 | Ralston John D | System, method and apparatus of video processing and applications |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4813056A (en) | 1987-12-08 | 1989-03-14 | General Electric Company | Modified statistical coding of digital signals |
US5574449A (en) | 1995-02-24 | 1996-11-12 | Intel Corporation | Signal processing with hybrid variable-length and entropy encodidng |
US6121904A (en) | 1998-03-12 | 2000-09-19 | Liquid Audio, Inc. | Lossless data compression with low complexity |
US6696922B1 (en) | 1999-08-18 | 2004-02-24 | 1229937 Ontario Limited | Interactive pager docking system |
KR20040065479A (en) * | 2003-01-14 | 2004-07-22 | 삼성전자주식회사 | System and method for editing multimedia file using internet |
DE10338696A1 (en) * | 2003-08-22 | 2005-03-17 | Siemens Ag | Method for creating and playing a media file |
US20050096025A1 (en) * | 2003-10-30 | 2005-05-05 | Iyad Qumei | Open update framework for mobile handsets capable of processing firmware and software update packages based on mime-types |
US7181538B2 (en) * | 2003-11-14 | 2007-02-20 | Sybase 365, Inc. | System and method for providing configurable, dynamic multimedia message service pre-transcoding |
US7606427B2 (en) * | 2004-07-08 | 2009-10-20 | Qualcomm Incorporated | Efficient rate control techniques for video encoding |
US20070260615A1 (en) * | 2006-05-08 | 2007-11-08 | Eran Shen | Media with Pluggable Codec |
-
2006
- 2006-02-16 US US11/357,661 patent/US20060218482A1/en not_active Abandoned
-
2011
- 2011-02-28 US US13/037,296 patent/US8849964B2/en not_active Expired - Fee Related
-
2014
- 2014-07-24 US US14/339,625 patent/US20140369671A1/en not_active Abandoned
Patent Citations (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6121998A (en) * | 1992-02-19 | 2000-09-19 | 8×8, Inc. | Apparatus and method for videocommunicating having programmable architecture permitting data revisions |
US6124882A (en) * | 1992-02-19 | 2000-09-26 | 8×8, Inc. | Videocommunicating apparatus and method therefor |
US6029196A (en) * | 1997-06-18 | 2000-02-22 | Netscape Communications Corporation | Automatic client configuration system |
US6847403B1 (en) * | 1997-11-05 | 2005-01-25 | Polycom, Inc. | Integrated portable videoconferencing unit |
US6356543B2 (en) * | 1997-11-25 | 2002-03-12 | Telefonaktiebolaget Lm Ericsson (Publ) | Controlling mobile phone system user views from the world-wide web |
US6167567A (en) * | 1998-05-05 | 2000-12-26 | 3Com Corporation | Technique for automatically updating software stored on a client computer in a networked client-server environment |
US6876379B1 (en) * | 1999-11-24 | 2005-04-05 | Orange Personal Communications Services Limited | Mobile communications |
US6754894B1 (en) * | 1999-12-03 | 2004-06-22 | Command Audio Corporation | Wireless software and configuration parameter modification for mobile electronic devices |
US20020184643A1 (en) * | 1999-12-16 | 2002-12-05 | Laurent Fichet | Tansmission of a command to a receiver or to a decoder |
US20020012347A1 (en) * | 2000-02-03 | 2002-01-31 | Patrick Fitzpatrick | System and method for downloading code |
US20010030667A1 (en) * | 2000-04-10 | 2001-10-18 | Kelts Brett R. | Interactive display interface for information objects |
US20020022522A1 (en) * | 2000-08-09 | 2002-02-21 | Konami Corporation | Game item providing system, method, and computer data signal |
US20020100036A1 (en) * | 2000-09-22 | 2002-07-25 | Patchlink.Com Corporation | Non-invasive automatic offsite patch fingerprinting and updating system and method |
US20020143792A1 (en) * | 2001-03-27 | 2002-10-03 | Sabin Belu | Systems and methods for creating self-extracting files |
US20040166834A1 (en) * | 2001-07-12 | 2004-08-26 | Omar Salim H. | System and method for providing remote data access and transcoding for a mobile communication device |
US20030084283A1 (en) * | 2001-09-04 | 2003-05-01 | Pixton Jeffrey Seth | Digital broadcast system |
US20040148362A1 (en) * | 2001-11-02 | 2004-07-29 | Lee Friedman | Systems and methods for managing and aggregating media formats |
US7266611B2 (en) * | 2002-03-12 | 2007-09-04 | Dilithium Networks Pty Limited | Method and system for improved transcoding of information through a telecommunication network |
US20040133914A1 (en) * | 2003-01-03 | 2004-07-08 | Broadq, Llc | Digital media system and method therefor |
US20060072837A1 (en) * | 2003-04-17 | 2006-04-06 | Ralston John D | Mobile imaging application, device architecture, and service platform architecture |
US20050038892A1 (en) * | 2003-08-13 | 2005-02-17 | Sun Microsystems, Inc. | Method for wireless communication and apparatus for conducting the same |
US20050036034A1 (en) * | 2003-08-15 | 2005-02-17 | Rea David D. | Apparatus for communicating over a network images captured by a digital camera |
US20080072261A1 (en) * | 2006-06-16 | 2008-03-20 | Ralston John D | System, method and apparatus of video processing and applications |
Cited By (172)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8896717B2 (en) * | 2002-04-19 | 2014-11-25 | Soryn Technologies Llc | Methods for deploying video monitoring applications and services across heterogeneous networks |
US20060085534A1 (en) * | 2002-04-19 | 2006-04-20 | Ralston John D | Video monitoring application, device architectures, and system architecture |
US20110113453A1 (en) * | 2002-04-19 | 2011-05-12 | Droplet Technology, Inc. | Methods for Displaying Video Monitoring Applications and Services Across Heterogeneous Networks |
US20130242119A1 (en) * | 2002-04-19 | 2013-09-19 | Vivox | Methods for Displaying Video Monitoring Applications and Services Across Heterogeneous Networks |
US7679649B2 (en) | 2002-04-19 | 2010-03-16 | Ralston John D | Methods for deploying video monitoring applications and services across heterogenous networks |
US20140368672A1 (en) * | 2002-04-19 | 2014-12-18 | Soryn Technologies Llc | Methods for Deploying Video Monitoring Applications and Services Across Heterogeneous Networks |
US20110181686A1 (en) * | 2003-03-03 | 2011-07-28 | Apple Inc. | Flow control |
US20050125733A1 (en) * | 2003-12-05 | 2005-06-09 | Ati Technologies, Inc. | Method and apparatus for multimedia display in a mobile device |
US7861007B2 (en) * | 2003-12-05 | 2010-12-28 | Ati Technologies Ulc | Method and apparatus for multimedia display in a mobile device |
US20050226170A1 (en) * | 2004-04-07 | 2005-10-13 | Sandeep Relan | System, method, and apparatus for secure sharing of multimedia content across several electronic devices |
US9912983B2 (en) | 2004-07-16 | 2018-03-06 | Virginia Innovation Sciences, Inc | Method and system for efficient communication |
US9286853B2 (en) | 2004-07-16 | 2016-03-15 | Virginia Innovation Sciences, Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US8712471B2 (en) | 2004-07-16 | 2014-04-29 | Virginia Innovation Sciences, Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US8903451B2 (en) | 2004-07-16 | 2014-12-02 | Virginia Innovation Sciences, Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US10104425B2 (en) | 2004-07-16 | 2018-10-16 | Virginia Innovation Sciences, Inc | Method and system for efficient communication |
US8948814B1 (en) | 2004-07-16 | 2015-02-03 | Virginia Innovation Sciences Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US9118794B2 (en) | 2004-07-16 | 2015-08-25 | Virginia Innovation Sciences, Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US8805358B2 (en) | 2004-07-16 | 2014-08-12 | Virginia Innovation Sciences, Inc. | Method and apparatus for multimedia communications with different user terminals |
US9589531B2 (en) | 2004-07-16 | 2017-03-07 | Virginia Innovation Sciences, Inc. | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US9355611B1 (en) | 2004-07-16 | 2016-05-31 | Virginia Innovation Sciences, Inc | Methods, systems and apparatus for displaying the multimedia information from wireless communication networks |
US9942798B2 (en) | 2004-07-16 | 2018-04-10 | Virginia Innovation Sciences, Inc. | Method and system for efficient communication |
US10136179B2 (en) | 2004-07-16 | 2018-11-20 | Virginia Innovation Sciences, Inc | Method and system for efficient communication |
US11109094B2 (en) | 2004-07-16 | 2021-08-31 | TieJun Wang | Method and system for efficient communication |
US10469898B2 (en) | 2004-07-16 | 2019-11-05 | Innovation Sciences, Llc | Method and system for efficient communication |
US10368125B2 (en) | 2004-07-16 | 2019-07-30 | Innovation Science LLC | Method and system for efficient communication |
US9729918B2 (en) | 2004-07-16 | 2017-08-08 | Virginia Innovation Sciences, Inc. | Method and system for efficient communication |
US20090171984A1 (en) * | 2004-10-30 | 2009-07-02 | Deuk Hee Park | Apparatus and method for interlinking the classified objects having the property and computer readable medium processing the method |
US20060170232A1 (en) * | 2005-01-25 | 2006-08-03 | Hammond Virgil E Iii | CINECOACH production systems |
US20060229029A1 (en) * | 2005-04-07 | 2006-10-12 | Intel Corporation | Ultra high frequency / very high frequency (UHF/VHF) band enhancement |
US20070288970A1 (en) * | 2006-05-11 | 2007-12-13 | Mats Tedenvall | Method and apparatus for sharing tv content with a remote device |
US20080059618A1 (en) * | 2006-08-30 | 2008-03-06 | Research In Motion Limited, | Automatic attachment of image and/or audio records to electronic calendar meeting event record in portable wireless devices |
US20080059648A1 (en) * | 2006-09-01 | 2008-03-06 | Freedom Broadcast Network, Llc | Dynamically Configurable Processing System |
US7809816B2 (en) * | 2006-09-19 | 2010-10-05 | At&T Intellectual Property I, L.P. | Imaging viewer systems and methods |
US20080071895A1 (en) * | 2006-09-19 | 2008-03-20 | Bellsouth Intellectual Property Corporation | Imaging viewer systems and methods |
US11196802B2 (en) | 2007-01-31 | 2021-12-07 | Samsung Electronics Co., Ltd. | Contents management method and apparatus |
US20080182558A1 (en) * | 2007-01-31 | 2008-07-31 | Samsung Electronics Co., Ltd. | Contents management method and apparatus |
US20100281042A1 (en) * | 2007-02-09 | 2010-11-04 | Novarra, Inc. | Method and System for Transforming and Delivering Video File Content for Mobile Devices |
US20080195698A1 (en) * | 2007-02-09 | 2008-08-14 | Novarra, Inc. | Method and System for Transforming and Delivering Video File Content for Mobile Devices |
US20080198264A1 (en) * | 2007-02-16 | 2008-08-21 | Nikhil Balram | Methods and systems for improving low resolution and low frame rate video |
US8885099B2 (en) * | 2007-02-16 | 2014-11-11 | Marvell World Trade Ltd. | Methods and systems for improving low resolution and low frame rate video |
US20080256169A1 (en) * | 2007-04-12 | 2008-10-16 | Hubert Oehm | Graphics for limited resolution display devices |
US20080260030A1 (en) * | 2007-04-17 | 2008-10-23 | Qualcomm Incorporated | Directional transforms for intra-coding |
US8238428B2 (en) | 2007-04-17 | 2012-08-07 | Qualcomm Incorporated | Pixel-by-pixel weighting for intra-frame coding |
US8937998B2 (en) | 2007-04-17 | 2015-01-20 | Qualcomm Incorporated | Pixel-by-pixel weighting for intra-frame coding |
US20080260031A1 (en) * | 2007-04-17 | 2008-10-23 | Qualcomm Incorporated | Pixel-by-pixel weighting for intra-frame coding |
US8406299B2 (en) | 2007-04-17 | 2013-03-26 | Qualcomm Incorporated | Directional transforms for intra-coding |
US8488672B2 (en) | 2007-04-17 | 2013-07-16 | Qualcomm Incorporated | Mode uniformity signaling for intra-coding |
US20080313686A1 (en) * | 2007-06-13 | 2008-12-18 | Matvey Thomas R | Handheld camcorder accessory with pre-programmed wireless internet access for simplified webcasting and handheld camcorder with built-in pre-programmed wireless internet access for simplified webcasting and method of commercially supplying and supporting same |
US10405021B2 (en) * | 2007-07-12 | 2019-09-03 | At&T Intellectual Property I, L.P. | System for presenting media services |
US7961747B2 (en) * | 2007-07-30 | 2011-06-14 | Broadcom Corporation | Display device with conversion capability for portable media player |
US20090034606A1 (en) * | 2007-07-30 | 2009-02-05 | Macinnis Alexander G | Display device with conversion capability for portable media player |
US20090150350A1 (en) * | 2007-12-05 | 2009-06-11 | O2Micro, Inc. | Systems and methods of vehicle entertainment |
US20090177476A1 (en) * | 2007-12-21 | 2009-07-09 | May Darrell | Method, system and mobile device for registering voice data with calendar events |
US20170293900A1 (en) * | 2008-01-09 | 2017-10-12 | T-Mobile Usa, Inc. | Mobile application monitoring system |
US9100389B2 (en) | 2008-10-21 | 2015-08-04 | Lookout, Inc. | Assessing an application based on application data associated with the application |
US8875289B2 (en) | 2008-10-21 | 2014-10-28 | Lookout, Inc. | System and method for preventing malware on a mobile communication device |
US8561144B2 (en) | 2008-10-21 | 2013-10-15 | Lookout, Inc. | Enforcing security based on a security state assessment of a mobile device |
US20110047620A1 (en) * | 2008-10-21 | 2011-02-24 | Lookout, Inc., A California Corporation | System and method for server-coupled malware prevention |
US9860263B2 (en) | 2008-10-21 | 2018-01-02 | Lookout, Inc. | System and method for assessing data objects on mobile communications devices |
US20110047594A1 (en) * | 2008-10-21 | 2011-02-24 | Lookout, Inc., A California Corporation | System and method for mobile communication device application advisement |
US10509911B2 (en) | 2008-10-21 | 2019-12-17 | Lookout, Inc. | Methods and systems for conditionally granting access to services based on the security state of the device requesting access |
US9779253B2 (en) | 2008-10-21 | 2017-10-03 | Lookout, Inc. | Methods and systems for sharing risk responses to improve the functioning of mobile communications devices |
US8683593B2 (en) | 2008-10-21 | 2014-03-25 | Lookout, Inc. | Server-assisted analysis of data for a mobile device |
US9781148B2 (en) | 2008-10-21 | 2017-10-03 | Lookout, Inc. | Methods and systems for sharing risk responses between collections of mobile communications devices |
US8510843B2 (en) | 2008-10-21 | 2013-08-13 | Lookout, Inc. | Security status and information display system |
US20110047597A1 (en) * | 2008-10-21 | 2011-02-24 | Lookout, Inc., A California Corporation | System and method for security data collection and analysis |
US8745739B2 (en) | 2008-10-21 | 2014-06-03 | Lookout, Inc. | System and method for server-coupled application re-analysis to obtain characterization assessment |
US9996697B2 (en) | 2008-10-21 | 2018-06-12 | Lookout, Inc. | Methods and systems for blocking the installation of an application to improve the functioning of a mobile communications device |
US8752176B2 (en) | 2008-10-21 | 2014-06-10 | Lookout, Inc. | System and method for server-coupled application re-analysis to obtain trust, distribution and ratings assessment |
US9740852B2 (en) | 2008-10-21 | 2017-08-22 | Lookout, Inc. | System and method for assessing an application to be installed on a mobile communications device |
US9223973B2 (en) | 2008-10-21 | 2015-12-29 | Lookout, Inc. | System and method for attack and malware prevention |
US9235704B2 (en) | 2008-10-21 | 2016-01-12 | Lookout, Inc. | System and method for a scanning API |
US8505095B2 (en) | 2008-10-21 | 2013-08-06 | Lookout, Inc. | System and method for monitoring and analyzing multiple interfaces and multiple protocols |
US8533844B2 (en) | 2008-10-21 | 2013-09-10 | Lookout, Inc. | System and method for security data collection and analysis |
US8826441B2 (en) | 2008-10-21 | 2014-09-02 | Lookout, Inc. | Event-based security state assessment and display for mobile devices |
US8984628B2 (en) | 2008-10-21 | 2015-03-17 | Lookout, Inc. | System and method for adverse mobile application identification |
US11080407B2 (en) | 2008-10-21 | 2021-08-03 | Lookout, Inc. | Methods and systems for analyzing data after initial analyses by known good and known bad security components |
US10509910B2 (en) | 2008-10-21 | 2019-12-17 | Lookout, Inc. | Methods and systems for granting access to services based on a security state that varies with the severity of security events |
US8881292B2 (en) | 2008-10-21 | 2014-11-04 | Lookout, Inc. | Evaluating whether data is safe or malicious |
US9407640B2 (en) | 2008-10-21 | 2016-08-02 | Lookout, Inc. | Assessing a security state of a mobile communications device to determine access to specific tasks |
US10417432B2 (en) | 2008-10-21 | 2019-09-17 | Lookout, Inc. | Methods and systems for blocking potentially harmful communications to improve the functioning of an electronic device |
US8381303B2 (en) | 2008-10-21 | 2013-02-19 | Kevin Patrick Mahaffey | System and method for attack and malware prevention |
US8365252B2 (en) | 2008-10-21 | 2013-01-29 | Lookout, Inc. | Providing access levels to services based on mobile device security state |
US9367680B2 (en) | 2008-10-21 | 2016-06-14 | Lookout, Inc. | System and method for mobile communication device application advisement |
US8099472B2 (en) | 2008-10-21 | 2012-01-17 | Lookout, Inc. | System and method for a mobile cross-platform software system |
US8347386B2 (en) | 2008-10-21 | 2013-01-01 | Lookout, Inc. | System and method for server-coupled malware prevention |
US8271608B2 (en) | 2008-10-21 | 2012-09-18 | Lookout, Inc. | System and method for a mobile cross-platform software system |
US9344431B2 (en) | 2008-10-21 | 2016-05-17 | Lookout, Inc. | System and method for assessing an application based on data from multiple devices |
US8997181B2 (en) | 2008-10-21 | 2015-03-31 | Lookout, Inc. | Assessing the security state of a mobile communications device |
US9043919B2 (en) | 2008-10-21 | 2015-05-26 | Lookout, Inc. | Crawling multiple markets and correlating |
US9294500B2 (en) | 2008-10-21 | 2016-03-22 | Lookout, Inc. | System and method for creating and applying categorization-based policy to secure a mobile communications device from access to certain data objects |
US9065846B2 (en) | 2008-10-21 | 2015-06-23 | Lookout, Inc. | Analyzing data gathered through different protocols |
US9245119B2 (en) | 2008-10-21 | 2016-01-26 | Lookout, Inc. | Security status assessment using mobile device security information database |
US8150464B2 (en) * | 2008-12-22 | 2012-04-03 | At&T Intellectual Property I, L.P. | Method and apparatus for providing a mobile video blog service |
US10200760B2 (en) | 2008-12-22 | 2019-02-05 | Red Hat, Inc. | Method and apparatus for providing a mobile video blog service |
US9584865B2 (en) | 2008-12-22 | 2017-02-28 | At&T Intellectual Property I, L.P. | Method and apparatus for providing a mobile video blog service |
US9178961B2 (en) | 2008-12-22 | 2015-11-03 | At&T Intellectual Property I, L.P. | Method and apparatus for providing a mobile video blog service |
US20100161635A1 (en) * | 2008-12-22 | 2010-06-24 | Spondon Dey | Method and apparatus for providing a mobile video blog service |
US20100191780A1 (en) * | 2009-01-29 | 2010-07-29 | Colin Kroll | Uploading a Hierarchical File Structure Using a Web Service |
US8825007B2 (en) | 2009-02-17 | 2014-09-02 | Lookout, Inc. | Systems and methods for applying a security policy to a device based on a comparison of locations |
US10623960B2 (en) | 2009-02-17 | 2020-04-14 | Lookout, Inc. | Methods and systems for enhancing electronic device security by causing the device to go into a mode for lost or stolen devices |
US9179434B2 (en) | 2009-02-17 | 2015-11-03 | Lookout, Inc. | Systems and methods for locking and disabling a device in response to a request |
US9232491B2 (en) | 2009-02-17 | 2016-01-05 | Lookout, Inc. | Mobile device geolocation |
US9167550B2 (en) | 2009-02-17 | 2015-10-20 | Lookout, Inc. | Systems and methods for applying a security policy to a device based on location |
US9100925B2 (en) | 2009-02-17 | 2015-08-04 | Lookout, Inc. | Systems and methods for displaying location information of a device |
US9955352B2 (en) | 2009-02-17 | 2018-04-24 | Lookout, Inc. | Methods and systems for addressing mobile communications devices that are lost or stolen but not yet reported as such |
US9042876B2 (en) | 2009-02-17 | 2015-05-26 | Lookout, Inc. | System and method for uploading location information based on device movement |
US20100210240A1 (en) * | 2009-02-17 | 2010-08-19 | Flexilis, Inc. | System and method for remotely securing or recovering a mobile device |
US8635109B2 (en) | 2009-02-17 | 2014-01-21 | Lookout, Inc. | System and method for providing offers for mobile devices |
US8855601B2 (en) | 2009-02-17 | 2014-10-07 | Lookout, Inc. | System and method for remotely-initiated audio communication |
US8929874B2 (en) | 2009-02-17 | 2015-01-06 | Lookout, Inc. | Systems and methods for remotely controlling a lost mobile communications device |
US10419936B2 (en) | 2009-02-17 | 2019-09-17 | Lookout, Inc. | Methods and systems for causing mobile communications devices to emit sounds with encoded information |
US8538815B2 (en) | 2009-02-17 | 2013-09-17 | Lookout, Inc. | System and method for mobile device replacement |
US8682400B2 (en) | 2009-02-17 | 2014-03-25 | Lookout, Inc. | Systems and methods for device broadcast of location information when battery is low |
US8467768B2 (en) | 2009-02-17 | 2013-06-18 | Lookout, Inc. | System and method for remotely securing or recovering a mobile device |
US20110047033A1 (en) * | 2009-02-17 | 2011-02-24 | Lookout, Inc. | System and method for mobile device replacement |
US8774788B2 (en) | 2009-02-17 | 2014-07-08 | Lookout, Inc. | Systems and methods for transmitting a communication based on a device leaving or entering an area |
US8239772B2 (en) * | 2009-03-24 | 2012-08-07 | Fuhu Holdings, Inc. | Apparatus, system and method for a user profiled-based OS for mobile devices |
US20100325629A1 (en) * | 2009-03-24 | 2010-12-23 | Robb Fujioka | Apparatus, System and Method for a User Profiled-Based OS for Mobile Devices |
WO2010119310A1 (en) * | 2009-04-16 | 2010-10-21 | Sony Ericsson Mobile Communications Ab | Methods, electronic devices, and computer program products for encoding image files to facilitate browsing thereof |
US20100265347A1 (en) * | 2009-04-16 | 2010-10-21 | Sony Ericsson Mobile Communications Ab | Methods, electronic devices, and computer program products for encoding image files to facilitate browsing thereof |
US8397301B2 (en) | 2009-11-18 | 2013-03-12 | Lookout, Inc. | System and method for identifying and assessing vulnerabilities on a mobile communication device |
US20110119765A1 (en) * | 2009-11-18 | 2011-05-19 | Flexilis, Inc. | System and method for identifying and assessing vulnerabilities on a mobile communication device |
USRE47757E1 (en) | 2009-11-18 | 2019-12-03 | Lookout, Inc. | System and method for identifying and assessing vulnerabilities on a mobile communications device |
USRE48669E1 (en) | 2009-11-18 | 2021-08-03 | Lookout, Inc. | System and method for identifying and [assessing] remediating vulnerabilities on a mobile communications device |
USRE49634E1 (en) | 2009-11-18 | 2023-08-29 | Lookout, Inc. | System and method for determining the risk of vulnerabilities on a mobile communications device |
USRE46768E1 (en) | 2009-11-18 | 2018-03-27 | Lookout, Inc. | System and method for identifying and assessing vulnerabilities on a mobile communications device |
US20110151880A1 (en) * | 2009-12-21 | 2011-06-23 | Electronics And Telecommunications Research Institute | Fixed mobile convergence terminal using downloadable wideband voice codec, method thereof and method of providing wideband voice codec in call control server |
US8611951B2 (en) * | 2009-12-21 | 2013-12-17 | Electronics And Telecommunications Research Institute | Fixed mobile convergence terminal using downloadable wideband voice codec, method thereof and method of providing wideband voice codec in call control server |
US20140153906A1 (en) * | 2010-03-09 | 2014-06-05 | Yolanda Prieto | Video enabled digital devices for embedding user data in interactive applications |
US9319292B2 (en) | 2011-06-14 | 2016-04-19 | Lookout, Inc. | Client activity DNS optimization |
US8738765B2 (en) | 2011-06-14 | 2014-05-27 | Lookout, Inc. | Mobile device DNS optimization |
US20140079369A1 (en) * | 2011-06-23 | 2014-03-20 | Huawei Device Co., Ltd. | Video Recording Method for Mobile Terminal, and Related Apparatus and System |
US10181118B2 (en) | 2011-08-17 | 2019-01-15 | Lookout, Inc. | Mobile communications device payment method utilizing location information |
US8788881B2 (en) | 2011-08-17 | 2014-07-22 | Lookout, Inc. | System and method for mobile device push communications |
US9218164B2 (en) * | 2011-09-26 | 2015-12-22 | Norman Ortiz | System and method for mobile application development using mobile devices |
US20150007131A1 (en) * | 2011-09-26 | 2015-01-01 | Norman Ortiz | System and Method for mobile application development using mobile devices |
US10256979B2 (en) | 2012-06-05 | 2019-04-09 | Lookout, Inc. | Assessing application authenticity and performing an action in response to an evaluation result |
US10419222B2 (en) | 2012-06-05 | 2019-09-17 | Lookout, Inc. | Monitoring for fraudulent or harmful behavior in applications being installed on user devices |
US20130325952A1 (en) * | 2012-06-05 | 2013-12-05 | Cellco Partnership D/B/A Verizon Wireless | Sharing information |
US9215074B2 (en) | 2012-06-05 | 2015-12-15 | Lookout, Inc. | Expressing intent to control behavior of application components |
US9992025B2 (en) | 2012-06-05 | 2018-06-05 | Lookout, Inc. | Monitoring installed applications on user devices |
US9589129B2 (en) | 2012-06-05 | 2017-03-07 | Lookout, Inc. | Determining source of side-loaded software |
US9940454B2 (en) | 2012-06-05 | 2018-04-10 | Lookout, Inc. | Determining source of side-loaded software using signature of authorship |
US11336458B2 (en) | 2012-06-05 | 2022-05-17 | Lookout, Inc. | Evaluating authenticity of applications based on assessing user device context for increased security |
US9407443B2 (en) | 2012-06-05 | 2016-08-02 | Lookout, Inc. | Component analysis of software applications on computing devices |
US9408143B2 (en) | 2012-10-26 | 2016-08-02 | Lookout, Inc. | System and method for using context models to control operation of a mobile communications device |
US9769749B2 (en) | 2012-10-26 | 2017-09-19 | Lookout, Inc. | Modifying mobile device settings for resource conservation |
US8655307B1 (en) | 2012-10-26 | 2014-02-18 | Lookout, Inc. | System and method for developing, updating, and using user device behavioral context models to modify user, device, and application state, settings and behavior for enhanced user security |
US9208215B2 (en) | 2012-12-27 | 2015-12-08 | Lookout, Inc. | User classification based on data gathered from a computing device |
US9374369B2 (en) | 2012-12-28 | 2016-06-21 | Lookout, Inc. | Multi-factor authentication and comprehensive login system for client-server networks |
US8855599B2 (en) | 2012-12-31 | 2014-10-07 | Lookout, Inc. | Method and apparatus for auxiliary communications with mobile communications device |
US9424409B2 (en) | 2013-01-10 | 2016-08-23 | Lookout, Inc. | Method and system for protecting privacy and enhancing security on an electronic device |
US10740482B2 (en) * | 2013-01-12 | 2020-08-11 | Pro Softnet Corporation | Method for sharing multiple data items using a single URL |
US20160162700A1 (en) * | 2013-01-12 | 2016-06-09 | Pro Softnet Corporation | Method for sharing multiple data items using a single url |
US9313368B2 (en) * | 2013-01-14 | 2016-04-12 | Samsung Electronics Co., Ltd. | Method for compressing image data collected by camera and electronic device for supporting the method |
US20140198233A1 (en) * | 2013-01-14 | 2014-07-17 | Samsung Electronics Co., Ltd | Method for compressing image data collected by camera and electronic device for supporting the method |
US10990696B2 (en) | 2013-10-25 | 2021-04-27 | Lookout, Inc. | Methods and systems for detecting attempts to access personal information on mobile communications devices |
US9642008B2 (en) | 2013-10-25 | 2017-05-02 | Lookout, Inc. | System and method for creating and assigning a policy for a mobile communications device based on personal data |
US10452862B2 (en) | 2013-10-25 | 2019-10-22 | Lookout, Inc. | System and method for creating a policy for managing personal data on a mobile communications device |
US9753796B2 (en) | 2013-12-06 | 2017-09-05 | Lookout, Inc. | Distributed monitoring, evaluation, and response for multiple devices |
US10742676B2 (en) | 2013-12-06 | 2020-08-11 | Lookout, Inc. | Distributed monitoring and evaluation of multiple devices |
US10122747B2 (en) | 2013-12-06 | 2018-11-06 | Lookout, Inc. | Response generation after distributed monitoring and evaluation of multiple devices |
US10306229B2 (en) | 2015-01-26 | 2019-05-28 | Qualcomm Incorporated | Enhanced multiple transforms for prediction residual |
US10540494B2 (en) | 2015-05-01 | 2020-01-21 | Lookout, Inc. | Determining source of side-loaded software using an administrator server |
US11259183B2 (en) | 2015-05-01 | 2022-02-22 | Lookout, Inc. | Determining a security state designation for a computing device based on a source of software |
US10623774B2 (en) | 2016-03-22 | 2020-04-14 | Qualcomm Incorporated | Constrained block-level optimization and signaling for video coding tools |
US10218697B2 (en) | 2017-06-09 | 2019-02-26 | Lookout, Inc. | Use of device risk evaluation to manage access to services |
US11038876B2 (en) | 2017-06-09 | 2021-06-15 | Lookout, Inc. | Managing access to services based on fingerprint matching |
US10956372B2 (en) | 2017-08-23 | 2021-03-23 | Bank Of America Corporation | Image capturing and processing for legacy format integration |
US10554840B2 (en) * | 2018-02-07 | 2020-02-04 | Kyocera Document Solutions Inc. | Image forming apparatus and non-transitory computer readable recording medium |
US11323748B2 (en) | 2018-12-19 | 2022-05-03 | Qualcomm Incorporated | Tree-based transform unit (TU) partition for video coding |
Also Published As
Publication number | Publication date |
---|---|
US20140369671A1 (en) | 2014-12-18 |
US20120106933A1 (en) | 2012-05-03 |
US8849964B2 (en) | 2014-09-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8849964B2 (en) | Mobile imaging application, device architecture, service platform architecture and services | |
US8896717B2 (en) | Methods for deploying video monitoring applications and services across heterogeneous networks | |
US20060072837A1 (en) | Mobile imaging application, device architecture, and service platform architecture | |
US9225760B2 (en) | System, method and apparatus of video processing and applications | |
EP1800415A2 (en) | Mobile imaging application, device architecture, and service platform architecture | |
US20140368672A1 (en) | Methods for Deploying Video Monitoring Applications and Services Across Heterogeneous Networks | |
AU2006214055A1 (en) | Mobile imaging application, device architecture, service platform architecture and services | |
CA2583745A1 (en) | Video monitoring application, device architectures, and system architecture | |
EP2210366B1 (en) | Methods and systems for transferring multimedia content using an existing digital sound transfer protocol | |
JP2000059776A (en) | Video information distribution system, its method and storage medium | |
Đukanović et al. | Possibilities of using Wavelets for still image sending in mobile networks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: DROPLET TECHNOLOGY, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RALSTON, JOHN D.;SAUNDERS, STEVEN E.;KOLAROV, KRASIMIR D.;REEL/FRAME:017655/0513 Effective date: 20060508 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |