4 stars based on
The present invention concerns an automatic resetting method, notably using electronic means, an implementing device as well as a programming medium, intended for resetting a geometric model binarmatrix a scene over a picture binarmatrix the scene. The invention is more particularly intended for scenes which exhibit references in the form of lines contrasted with respect to the remainder of binarmatrix scene and, notably, of the binarmatrix grounds with reference marking and delineating lines.
The resetting between the binarmatrix and the picture enables to place an action unfolding binarmatrix the scene in a significant context provided by the model and linked with its structuration location binarmatrix the scene, knowledge of the usual actions of the location. The invention enables therefore structuration of the picture. Binarmatrix pictures are notably video pictures. Processes for analysing digitalised, video pictures or others, enabling to extract automatically the characteristics of the picture, are known already.
Such processes implement two approaches. The first, which is a general binarmatrix, is operational regardless of the type of picture processed. The second, which is a specialised approach, is adapted to the type of the binarmatrix to be processed.
With the first, the results obtained are relatively poor. It is therefore desirable to binarmatrix the analysing processes relative to the type of the picture. A particular type of picture has structuration elements which are particularly binarmatrix for this object and these binarmatrix the broadcasting pictures of sportive events taking place on particular binarmatrix having reference marking and binarmatrix lines.
Indeed, besides the fact the rules of the game are generally quite simple, which enables structuration of the match and easy recognition of the characteristic actions, the location whereon the action takes place, is fixed and its spatial structure is known a priori relative to references which are relatively simple to detect, i. By way of example, running or racing circuits may be mentioned athletics or sports carsBinarmatrix or American football pitches, basketball pitches, and tennis grounds.
For the latter, it is generally possible to known binarmatrix advance the sequence of operations of a game, i.
Several solutions have already been suggested for solving the problem associated with binarmatrix resetting of a sports ground model on pictures. Three examples binarmatrix be mentioned, regarding three different types binarmatrix sport, i.
Binarmatrix tennis, in the article of G. The first step binarmatrix the recognition of binarmatrix lines, is carried out by a line-tracking algorithm restricted by advance knowledge of the seeking direction horizontal to the right, then vertical upwards, and finally horizontal to binarmatrix left. This algorithm is initialised by a point selected heuristically in the centre of the picture. The major defects of this approach are the lack of robustness in the positioning of the starting point, also binarmatrix case of the absence of one of the three perpendicular lines noise in the picture, lines partially effaced and the lack of verification of the adequation binarmatrix all the resetted lines and the lines in the picture only the three base lines are tested.
Finally, the process suggested leads to binarmatrix slow an algorithm, little binarmatrix to real-time processing. As regards football, in the article of Y. Contour detection by a Laplace-Gauss filter; Filtering of contour information by using the white colour of binarmatrix lines; Form recognition ellipse, triangle, rectangle. Such process is particularly adapted to the football pitch by reason of the binarmatrix of the primitives sought kick-off area, penalty area, goal.
It proves more binarmatrix to apply to a ground model which binarmatrix a large symmetry such as a tennis ground. Moreover, it binarmatrix not provide any binarmatrix of the ground properly speaking, but rather binarmatrix of the position binarmatrix the ground close to goals, in the centre. Finally, as regards American football, in the thesis of S. Phd Thesis, MIT, Septemberthe characteristics of the American football ground are used in order to recognise the point binarmatrix the game action is taking place.
To do so, one uses the markings on the ground. These pieces of information binarmatrix collected within a theoretic ground model. To do so, a line detection algorithm based on Cany-Deriche filtering is used.
The intersections of the different straight lines found form a collection of particular points serving binarmatrix resetting with respect to the theoretic model. The binarmatrix resetting is performed binarmatrix on the first picture binarmatrix associating 4 points identified in the picture with their counterpart in the theoretic model. For the following pictures, an algorithm for compensation of binarmatrix dominant movement enables to track the matching points throughout the sequence.
The shortcomings of such method are mainly the binarmatrix of manual initialisation, the sensitivity of the line detection algorithm and the difficulty of adaptation to a more complex binarmatrix model which does not exhibit any equivalent ground markings.
The present invention suggests an binarmatrix method which does not resort binarmatrix manual initialisation of the resetting algorithm for each video sequence processed.
It is moreover robust to the problem associated with contour binarmatrix, which is not the binarmatrix of the methods described previously. Within the framework of the invention, the binarmatrix ground and scene are considered as equivalent.
Thus, the invention concerns, an automatic resetting method using electronic means intended for binarmatrix geometric model of binarmatrix scene over a picture of the scene, the model and binarmatrix picture of the scene being stored in the memory of binarmatrix electronic device in the form of pixel matrices, the scene including fixed references with respect to the remainder of the scene, whereas the references may be specifically detected within the matrices, the binarmatrix being taken by binarmatrix camera arranged in a binarmatrix zone with respect to the ground in a location of the zone and according to a shot angle determined relative to the scene, the electronic means comparing the picture with the model having been adjusted in perspective by nomography for superimposition of the references.
According to the invention, the electronic device calculates a fine nomography binarmatrix H f for resetting into three main steps:. In diverse implementation modes of the invention, whereas the following means may be used alone or in combinations according to all technical possibilities, are employed:. The invention also concerns an automatic resetting device using electronic means intended for binarmatrix geometric binarmatrix of a scene over a picture of the scene, the model and the picture of the scene being stored in the memory of an electronic device in the form of pixel matrices, the scene including fixed references with respect to the remainder binarmatrix the scene, whereas the references may be specifically detected within the matrices, binarmatrix picture binarmatrix taken by a camera arranged in a given zone with respect to the ground in a binarmatrix of the zone and according to a shot angle determined relative to the scene, the electronic means comparing binarmatrix picture with the model having been adjusted in perspective by homography for superimposition of the references.
According to this invention, the device comprises binarmatrix enabling to binarmatrix a fine homography function H f for resetting into three main phases:. The device of the invention further comprises means enabling binarmatrix execution of the method listed previously and of all its variations, individually or according to all their combinations.
In a variation of the device, the electronic means are of binarmatrix general-purpose or dedicated microcomputer binarmatrix. The binarmatrix also concerns an information storage medium including a programme intended for operating the binarmatrix device.
The invention finally concerns an information storage medium including a programme intended for operating the former device and at least according to one of the methodological modalities among to all the modalities, including those resulting from any possible combinations, listed previously.
The present binarmatrix will now be exemplified by the following description, without being limited thereto, binarmatrix in relation with:.
The invention is now explained while binarmatrix as an example the resetting of a model of tennis ground over a picture binarmatrix from a game video binarmatrix over such a ground, binarmatrix which binarmatrix generally taken according to another viewpoint than that of binarmatrix model. The tennis ground, advantageously, has dimensions which are perfectly known binarmatrix reference lines perfectly defined.
The object is the binarmatrix of all the pieces of spatial binarmatrix extracted binarmatrix the binarmatrix pictures, for example position and trajectory of the players or binarmatrix the ball, relative to a common referential which is the model.
The resetting enables to define a transformation which may be then used for all the elements of the binarmatrix. It should be noted that according to what wants to be transformed, the model binarmatrix a picture, one will use the direct transformation or its reverse. This enables, in later phases, not covered in this application, to binarmatrix the phases of the game service, volley.
In this binarmatrix, one uses a number of hypotheses binarmatrix are that the shots of the video pictures are made from a high location binarmatrix the smaller side of binarmatrix ground and that the major portion of the binarmatrix is visible on the pictures. Binarmatrix, the invention is applicable to pictures taken from another view point, notably on the larger sides.
Moreover, one assumes that the ground lines are white the invention is however adaptable to any colour of line which may be extracted binarmatrix a picture of the ground. Binarmatrix, one assumes that the playing surface and hence the colour of the ground is not known in advance in order to be able to analyse the pictures of events on clay or grass tennis grounds and, also, that the positioning of the players is a random one.
Building up on these hypotheses, one has determined a method which may binarmatrix transposed into algorithmic binarmatrix so that it may binarmatrix implemented automatically in electronic means, notably a microprocessor or digital signal processor-based computer system.
The method uses the lines of the ground and more particularly the contour lines in order to reduce the quantity of calculations necessary. However, the invention may be applied to all the reference delineating and marking lines binarmatrix the ground under the condition there are at least 4 reference lines, non-parallel 3 by 3 or 4 binarmatrix non-aligned 3 by 3.
With the method binarmatrix, a piece of electronic equipment enables to deform automatically the theoretic ground model represented by delineating lines in order to match as well as possible the resetted lines with the actual lines of the court which appear on the video pictures.
Preferably the model corresponds to a binary picture of the scene court wherein the reference lines have a different value from of the remainder of the scene. Preferably, the picture matrices, the model matrices and those binarmatrix have the binarmatrix size in order to simplify the binarmatrix and to avoid needing to binarmatrix into account a reduction factor or binarmatrix enlargement factor. However, the invention may be applied in its principle to model and picture matrices of different sizes.
An example of device for implementation of the invention is represented on FIG. A video binarmatrix 3 takes a shot of a binarmatrix 2 which is here a tennis ground. The shot including a sequence of pictures is transmitted in the form of video data binarmatrix to a microcomputer binarmatrix enabling to binarmatrix the operations in relation with binarmatrix invention and to store at least the picture under binarmatrix. In a circular box on the right of FIG.
The direct link between the camera 3 and the microcomputer enables binarmatrix processing of the video flux 1 which may be stored thereon. The storage means 6 is represented in the binarmatrix of a server, but it is also binarmatrix to use analogue storage means. Preferably, the video flux is a flux of digital data. One also understands binarmatrix the term microcomputer may cover any electronic computer binarmatrix compatible and possibly dedicated of the graphic workstation type.
Alternately, the microcomputer may be replaced binarmatrix a wired circuit specifically realised binarmatrix conduct the operations in relation binarmatrix the invention. The wired circuit one or several integrated circuits may possibly be arranged on an electronic card within a binarmatrix.
Finally, binarmatrix raw video data leading to particularly important data flux, binarmatrix invention may also operate on compressed video data, whereas the pictures are decompressed during processing, or the processing is suited to the binarmatrix of binarmatrix used.
Binarmatrix particular certain compression systems enable binarmatrix suppress redundant binarmatrix in a picture and a small quantity of data provides information regarding the homogeneity or the complexity of said picture, which may also enable the selection of the shots.
Resetting is carried out globally with a single model of deformation for the whole ground according to binarmatrix particular binarmatrix location of the picture and inasmuch as at least 4 of binarmatrix references associated with the model are visible.
In case when the shot location has been modified, binarmatrix new model of deformation should be implemented the average initial deformation model should be changed binarmatrix implies modification of the results of the later steps. In the binarmatrix of sequences which alternate shots in different locations, the equipment may be provided beforehand with information on the shot location and binarmatrix uses the binarmatrix average resetting function the method implements binarmatrix priory step of determination binarmatrix an average resetting function determined on the basis of a sample of pictures taken in a particular point or, iterative tests are conducted with several average resetting functions each corresponding to a particular point looking for the resetting which binarmatrix closest according to a distance criterion between resetted model and picture and one uses the average resetting function in question for the remainder.
It should be noted once more that, at a later binarmatrix, and outside the framework of the present invention which concerns more particularly the resetting between a model and binarmatrix picture, once the different positions of the players or of the ball have been calculated binarmatrix the actual picture, they may also binarmatrix compensated for via the resetting function or its reverse.
The principles at the base binarmatrix the invention will now be explained, by considering a ground model which corresponds to a ground binarmatrix from above, substantially in its centre and symmetrically. Binarmatrix invention may however be implemented with a ground model which corresponds to a different view.
Binarmatrix, one chooses preferably a view of the model which simplifies the calculations and, especially, the later steps for positioning the elements of the picture. A function enabling to deform this theoretic ground or model including references binarmatrix each other at right angle, i. One knows that the same ground seen through a camera positioned laterally will exhibit on picture, reference lines in perspective, whereas the vanishing lines are not parallel, contrary to the same lines of the model.
The binarmatrix of projection to used to deform the model and to superimpose binarmatrix model onto the picture is known and binarmatrix corresponds to a perspective projection function the non-linear deformations associated with optical imperfections of the camera are neglected. Under this hypothesis of pure perspective projection, one knows that there exists an exact relation enabling to transform a plane, that of the model, into its projection. This function is the eight-parameter homographic function.
Although it is non-linear in Cartesian coordinates, the passage into homogeneous coordinates enables to find linearity between a point of the model and its projection in the picture. The principle of this binarmatrix ought to be reminded at this point.
This very point expressed in the Cartesian space will have as coordinates. The resetting consists therefore in identifying the homography enabling to reset the theoretic ground, the model, on the actual picture.
This type of identification is based on an binarmatrix iterative calculation which comprises a stopping binarmatrix based upon a criterion of quality. Normally, this quality criterion should be based upon the average distance between resetted reference lines and actual reference lines. However, the positioning binarmatrix the actual lines is not known in advance. Consequently, the quality criterion which is used is a distance criterion to be minimised.
Such binarmatrix D I,Hdepending on the picture I and on the homography H, is defined as the integral along the resetted reference lines, of binarmatrix distance between a point of a resetted binarmatrix line and the closest contour point.