The data includes a total of 90 videos, including 72 videos with actions within standard operating procedures and 18 videos with actions that are not within standard operating procedures, totaling approximately 270,000 frames, with a total length of approximately three hours, and an average video length of approximately one hundred seconds.
With multiple large open source datasets, the development of action recognition is rapid. However, we noticed the lack of annotated data of cilvil aircraft pilots, while distribution of whose action can be very different from daily casual activities. After discussion with experienced pilots and experts and close look into standard operation procedure, we present Airline-Pilot-Action (APA) benchmark, containing 5090 RGB and depth images together with corresponding flight computer data.
This dataset consists of inertial, force, color, and LiDAR data collected from a novel sensor system. The system comprises three Inertial Measurement Units (IMUs) positioned on the waist and atop each foot, a color sensor on each outer foot, a LiDAR on the back of each shank, and a custom Force-Sensing Resistor (FSR) insole featuring 13 FSRs in each shoe. 20 participants wore this sensor system whilst performing 38 combinations of 11 activities on 9 different terrains, totaling over 7.8 hours of data.
In order to conform to our proposed network, we processed the skeleton data in NTU-RGB+D, which is divided into two files: skeleton.npy shaped as (52678,3,80,25,2), representing 52678 actions; skeleton_label.pkl shaped as (52678,2), representing 52678. The IDs of the actions and the original file names. Ids from 0 to 59 represent the 60 different types of actions. The raw file name is the action file with a skeleton extension.
This dataset was collected in our lab using Kinect to emphasize three points: (1) Larger number of human activities. (2) Each subject performed all actions in a continuous manner with no breaks or pauses. Therefore, the start and end positions of body for the same actions are different. (3) Each subject performed the same actions four times while imaged from four different views: front view, left and right side views, and top view.
