research paper on space debris

A Comprehensive Study on Space Debris, Threats Posed by Space Debris, and Removal Techniques

Proceedings of the Second International Conference on Emerging Trends in Science & Technologies For Engineering Systems (ICETSE-2019)

8 Pages Posted: 2 Jan 2020 Last revised: 23 Jul 2020

Sangita Mullick

S J C Institute of Technology, Department of Aeronautical Engineering, Students

Yashwanth Srinivasa

Ashutosh kumar sahu, jhanvi tharun sata.

Date Written: May 17, 2019

After exploring space for more than 50 years for research, study and defense purposes, the region above the atmosphere of earth is highly polluted by orbital debris. Figure 1 shows the total number of rocket launches in period of nine years. This has become a concern for placing satellites in their respective orbits and their safe functioning during their mission. Space debris or orbital debris colloquially known as space junk are parts of the non-functional satellites, thermal blankets, booster stages of the rockets. Those satellites are placed in the several orbits according to their missions. Mainly, they are placed in LEO (Low Earth Orbit), an earth centered orbit ranging from 200 to 2000 kilometers. Some are also placed in GEO (Geostationary Earth Orbit), at an altitude of 36000 kilometers and some are placed in the Higher Earth Orbit. Since the dawn of space age, approximately 7000 rockets have been launched, placing their payloads in several orbits of the Earth, revolving at several kilometers per second. And more than half of these objects are present in LEO. It is estimated that their sizes vary from a few millimeters to few meters, the largest being the European Envisat. Because of their high speeds, pieces of debris not more than a millimeter apart also poses a huge risk to current and upcoming space missions. Since the risk is increasing exponentially and is of great concern for all the space-faring nations, there is a need for the active removal of space debris. Hence, in this paper, the authors have analyzed the threat that space debris poses, and some of its removal techniques that have been proposed by scientists and space organizations. The authors have also suggested a few more of these Active Debris Removal techniques.

Keywords: Space Debris, Threats posed by Space Debris, and Removal Techniques

Suggested Citation: Suggested Citation

Sangita Mullick (Contact Author)

S j c institute of technology, department of aeronautical engineering, students ( email ), do you have a job opening that you would like to promote on ssrn, paper statistics, related ejournals, industrial & manufacturing engineering ejournal.

Subscribe to this fee journal for more curated articles on this topic

Aerospace Engineering eJournal

Astronomy & astrophysics ejournal, materials performance ejournal, computational materials science ejournal.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Journal Menu

• Aerospace Home
• Aims & Scope
• Editorial Board
• Reviewer Board
• Topical Advisory Panel
• Instructions for Authors
• Special Issues
• Sections & Collections
• Article Processing Charge
• Indexing & Archiving
• Most Cited & Viewed
• Journal Statistics
• Journal History
• Journal Awards
• Society Collaborations
• Conferences
• Editorial Office
• 10th Anniversary

Journal Browser

• arrow_forward_ios Forthcoming issue arrow_forward_ios Current issue
• Vol. 11 (2024)
• Vol. 10 (2023)
• Vol. 9 (2022)
• Vol. 8 (2021)
• Vol. 7 (2020)
• Vol. 6 (2019)
• Vol. 5 (2018)
• Vol. 4 (2017)
• Vol. 3 (2016)
• Vol. 2 (2015)
• Vol. 1 (2014)

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

Space Debris Removal: Challenges and Opportunities

• Print Special Issue Flyer
• Special Issue Editors

Special Issue Information

Benefits of publishing in a special issue.

• Published Papers

A special issue of Aerospace (ISSN 2226-4310). This special issue belongs to the section " Astronautics & Space Science ".

Deadline for manuscript submissions: closed (30 June 2022) | Viewed by 25856

Share This Special Issue

Special issue editor.

Dear Colleagues,

Over the past few decades, Earth orbits, specifically the geosynchronous orbits ideal for communication satellites and the Sun synchronous orbits favored for Earth observation satellites, are increasingly crowded with human-made space debris. Controlling the population of space debris is commonly recognized as a critical task for the safety of operating satellites and long-term sustainability of our space activities. The first ever satellite–satellite collision between operational and abandoned satellites in 2009 is a just wake-up call. While most current efforts focus on debris mitigation methods and strategies, it is widely believed that the population of space debris will continue to grow over time unless we actively remove five or more massive pieces of debris from the orbit annually.

In this Special Issue, we invite high-quality original contributions covering all aspects of space debris removal—the current challenges, methodologies, and opportunities. Papers dealing with new technology developments for passive/active space debris removal technologies and strategies, the associated technological readiness of solutions, analysis and/or experimental results in the context of space debris removal methodologies, business perspectives, and initiatives are welcome. 

Prof. Dr. George Z.H. Zhu Guest Editor

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website . Once you are registered, click here to go to the submission form . Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Aerospace is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

• space debris
• dynamic modeling
• debris detection, surveillance, and tracking
• collision avoidance
• approach and capture of space debris
• passive and active removal techniques and strategies
• risk analysis and management
• autonomous operation and control
• Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
• Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
• Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
• External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
• e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here .

Published Papers (8 papers)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 19 December 2022

Localization and classification of space objects using EfficientDet detector for space situational awareness

• Nouar AlDahoul 1 , 3 ,
• Hezerul Abdul Karim 1 ,
• Angelo De Castro 2 &
• Myles Joshua Toledo Tan 2  

Scientific Reports volume  12 , Article number:  21896 ( 2022 ) Cite this article

4085 Accesses

8 Citations

3 Altmetric

Metrics details

• Computational science
• Computer science
• Information technology

Space situational awareness (SSA) systems play a significant role in space navigation missions. One of the most essential tasks of this system is to recognize space objects such as spacecrafts and debris for various purposes including active debris removal, on-orbit servicing, and satellite formation. The complexity of object recognition in space is due to several sensing conditions, including the variety of object sizes with high contrast, low signal-to-noise ratio, noisy backgrounds, and several orbital scenarios. Existing methods have targeted the classification of images containing space objects with complex backgrounds using various convolutional neural networks. These methods sometimes lose attention on the objects in these images, which leads to misclassification and low accuracy. This paper proposes a decision fusion method that involves training an EfficientDet model with an EfficientNet-v2 backbone to detect space objects. Furthermore, the detected objects were augmented by blurring and by adding noise, and were then passed into the EfficientNet-B4 model for training. The decisions from both models were fused to find the final category among 11 categories. The experiments were conducted by utilizing a recently developed space object dataset (SPARK) generated from realistic space simulation environments. The dataset consists of 11 categories of objects with 150,000 RGB images and 150,000 depth images. The proposed object detection solution yielded superior performance and its feasibility for use in real-world SSA systems was demonstrated. Results show significant improvement in accuracy (94%), and performance metric (1.9223%) for object classification and in mean precision (78.45%) and mean recall (92.00%) for object detection.

Similar content being viewed by others

RGB-D based multi-modal deep learning for spacecraft and debris recognition

A streamlined approach for intelligent ship object detection using EL-YOLO algorithm

A Dataset for Detection and Segmentation of Underwater Marine Debris in Shallow Waters

Introduction.

Over the past few decades, operations carried out by space organizations, such as the National Aeronautics and Space Administration (NASA) and the European Space Agency (ESA), have resulted in enormous amounts of space debris being sent into orbit around the planet. Space agencies' operations are mostly focused on the navigation of the solar system, weather monitoring on Earth, and space launch campaigns, to name a few examples. The Space Situational Awareness Initiative (SSA) aims to equip Europe and its residents with full and reliable data on objects circling the Earth, and on the dangers originating in space. By fulfilling its objectives, the SSA program will allow Europe to independently identify, anticipate, and evaluate the dangers to people and property posed by various perils that could happen in the solar system 1 .

In recent years, several works have been presented to study the possible benefits of artificial intelligence (AI), particularly deep learning (DL), on improving accuracy of the classification and detection of objects in photographs taken for space operations 2 , 3 , 4 , 5 , 6 , 7 . The accessibility and quality of information needed to train deep learning systems have a significant impact on their efficiency, as has been demonstrated in various studies 8 , 9 , 10 . Data, on the other hand, are extremely rare and expensive to gather in the space domain. In addition, various research has been conducted utilizing vision-based sensors, and to perform unsupervised near-earth missions in space with refractory targets, and durable and fast onboard posture stabilization techniques that are necessary on the spacecraft 11 , 12 , 13 . As a result, various image-based studies suggest the use of motion sensors such as Light Detection and Ranging (LiDAR) to achieve this goal 14 , 15 , 16 . To utilize neural network models successfully, a lot of data are needed. Monitoring the encircling components around the spacecraft is challenging since they vary in size, shape, and composition. Obtaining data from these spacecrafts is also a costly task. For this reason, research teams have begun to examine satellite data collection. The SPARK dataset offers a realistic representation of the earth and of the objects in and around it 14 , 17 , 18 .

Various object recognition algorithms have been published over the last decade, and it is fascinating to explore the suitability of these techniques to space data, as well as to find ways for improving their efficiency in the space domain. Unsupervised object detection is performed by convolutional neural networks (CNNs), which eliminate the need for features to be generated and obtained individually 19 . Deep learning-based techniques use CNNs to do this task.

Related works

The region-based convolutional neural network (R-CNN) family of object identification algorithms includes a variety of widely used object detection techniques 20 . Premised on the region proposal architecture, which is an extended version of the linear regression technique and is also used by Faster R-CNN 21 , these frameworks have been designed and implemented. It is believed that this decoder will find objects in portions of an image where the algorithm anticipates they may be present 22 . As technology progresses, algorithms, likewise, become more precise but also become more computationally expensive. Mask R-CNN, designed by developers at Facebook, is one of the most recent algorithms that serves as a useful initial point for object detection models on the client side of the network 23 . On the other hand, single shot detectors (SSDs) are designed to depend on a fixed number of regions instead of a subnetwork to suggest regions. Upon overlaying an input image, a grid of reference points is created, and at each point, boxes of various shapes and sizes are used to define the areas 24 . There are also a variety of versions available that are part of the single shot detector network. The encoders used in each model, as well as the precise layout of predefined points, are the primary differences between them. The MobileNet + SSD models include a MobileNet-based encoder 25 , while the YOLO model includes a convolutional architecture that is proprietary to it. The YOLO concept takes a completely distinct approach. It uses a single neural network to process the entire picture. For each region, this network separates the picture into regions, from which it anticipates the bounding boxes and probabilities 26 , 27 , 28 . Thus, the weighting of these bounding boxes is determined by the projected probability. For this reason, SSDs are excellent alternatives for models that could be used in mobile or embedded systems. Furthermore, in a recent publication, the Google Brain team described their EfficientDet architecture for object detection, intending to design selections into a scalable structure that can be used for a variety of diverse object detection applications 29 . On standard datasets, the study suggests that EfficientDet performs better simulations of equal size.

To reduce the possibility of collisions occurring in space, the process of target recognition ought to be carried out automatically 30 . The most important component in SSA for analyzing visual data and developing data-driven AI solutions is the vision-based sensor 13 , 14 , 15 , 31 , which can take the form of a camera. On one hand, past research papers have presented a variety of technologies to detect and manage active and inactive satellites, while on the other hand, several strategies have been suggested to eliminate debris from space. In addition, LiDAR sensors have been utilized for the removal of debris, the recognition of targets, and the estimation of poses 13 , 14 , 15 , 16 . It was discovered that there are ways for estimating the pose of a 3D spacecraft by comparing the wireframe of the target with a 2D image. These approaches make use of a matching process that compares visual elements taken from the image and the wireframe 5 . In order to find the pose, the Perspective-n-Point (PnP) argument needed to be solved 5 . In order to extract the edge characteristics, traditional computer vision methods such as Sobel and Canny detectors were utilized 32 , 33 . On the other hand, conventional machine learning algorithms were taken into consideration for the task of posture estimation utilizing principal component analysis (PCA) 34 . After applying PCA to a spacecraft image in question, the results were compared with the ground truth postures contained inside the dataset for the objective of matching. Detecting objects, determining their bounding boxes, and classifying images are some of the most important challenges in computer vision. Object detection and image classification are used to accomplish these goals. Deep learning techniques, which use automatic feature learning and extraction, have been shown to generate superior outcomes over other computer vision methods. As a consequence, deep learning algorithms have been implemented in space applications with the goal of recognizing spacecraft and debris for a variety of reasons. One of the deep learning models that were utilized to estimate the posture of the spacecraft was a pre-trained convolutional neural network 6 , 35 , such as GoogleNet 7 , 36 . Similarly, VGG 37 , 38 has been trained and evaluated on a synthetic dataset in order to identify the translation and rotation of a space object relative to a camera. In addition, ResNet was presented for the purpose of estimating the pose of an uncooperative spacecraft without the use of any 3D input and for predicting the bounding box of space objects 5 , 10 . The quantity of information that is input into the deep learning model is directly related to both the effectiveness of the method and its capacity for generalization. In order to achieve the desired level of efficiency in comparison to more conventional machine learning strategies, the size of the data set used must be substantial. The expense of acquiring data from spacecrafts is quite high. As a result of this, many different synthetic datasets have been presented in research works for the purpose of 6D pose estimation. Two examples of these are the Unreal Rendered Spacecraft On-Orbit (URSO) dataset 6 and the Spacecraft posture estimation dataset (SPEED) 39 , 40 . The fact that the nearby spacecraft or objects are of varying sizes makes object surveillance a difficult and complicated process. This is in addition to the high cost of acquiring space data. Researchers have taken into consideration the technique of data gathering to collect images of space objects such as spacecraft and debris in order to solve the issues that were previously identified. Because of this, they created a high-resolution synthetic spacecraft dataset by using the environment simulator that comes with the Unity3D gaming engine 41 . In order to provide an adequately labelled space dataset, a new SPARK dataset was put together and was designed exclusively for the classification of space objects 14 , 18 . The SPARK dataset portrays a genuine earth and other objects located in its immediate vicinity. Both ResNet 10 and EfficientNet 42 were presented as examples of pre-trained CNNs that made use of the SPARK dataset and a number of different examples 14 . The three possible outcomes are as follows: (1) initializing the models with random data and beginning the training process from scratch; (2) feature extraction by freezing the backbone of the network and only training the classifiers in the top layers of the network; and (3) making use of the pre-trained weights and then fine-tuning on the entire model, including the backbone and the classifier. It was discovered that the algorithms that were trained on both RGB and depth pictures performed significantly better than single models 14 .

AlDahoul et al. 43 have proposed a multi-modal learning method with SPARK dataset. They formulated the problem as an image classification problem to identify the space object category directly from the whole image applied to the CNN. The features were extracted from RGB images of spacecraft and debris, utilizing numerous convolutional neural networks such as DenseNet, ResNet, and EfficientNet. They also explored vision transformer for same purpose. For depth images classification, the End-to-End CNN was demonstrated. They have found that combining RGB based vision transformer and depth-based End-to-End CNN produced better performance in terms of accuracy and F1 score.

On the other hand, localization of space objects before classification was proposed to focus attention on regions of space objects and to ignore other irrelevant objects in the background 2 . Their detection algorithm did not use traditional object detectors, such as YOLO and faster R-CNN, which require annotation with bounding boxes for objects in each image. They implemented a simple detection algorithm on depth images in a few steps: (1) smoothing images using a Gaussian filter; (2) up-sampling images twice to produce the depth images that have the same size as the RGB image; and (3) converting images to black-and-white by thresholding and inverting them. After obtaining cropped images that have only space objects in RGB and depth versions, a decision fusion approach was applied.

This study demonstrates the utilization of an object detection method using an EfficientDet model that has been found to outperform other object detectors for various applications 44 . The first objective is to enhance classification performance by focusing attention on regions of space objects and ignoring other irrelevant objects in the background. This contributes to the improvement of accuracy and performance metrics when compared with existing solutions. Moreover, localization of space objects in the image by predicting four coordinates of the object is the second significant objective that helps SSA systems in space navigation missions.

The study presented in this paper aims to attract the research community by highlighting an interesting new challenge that enriches the body of knowledge by proposing the following:

A space object detection model that localizes debris and spacecraft objects in RGB-based space images and that classifies them into eleven classes;

A multi-modal learning approach for spacecraft classification that uses only RGB images to combine decisions from efficientNet-v2 and EfficientNet-B4;

An evaluation of metrics and comparison with methods utilized for the same purpose of space object classification.

An ablation study to validate significant improvements in classification accuracy by using multi-modal learning, which yield the final decision by combining decisions from efficientNet-v2 and EfficientNet-B4 CNNs.

The organization of this paper is as follows: The description of the SPARK space imagery dataset was done in section “ Materials and methods ”. Furthermore, the approaches to object detection and multi-modal learning were also demonstrated in this section. Section “ Results and discussion ” discusses the experiments conducted in this study and analyses the results by comparing the proposed method with the existing solutions. Finally, in section “ Conclusion and future work ”, we summarized the outcome of this work to give the readers a glimpse into potential improvements in the future.

Materials and methods

The description of the dataset used in this work is presented in this section to highlight the challenging contents available in this dataset of space images. Furthermore, the object detection method of EfficientDet is demonstrated to shed light on its superiority over state-of-the-art object detectors. Additionally, a decision fusion approach is discussed to study the efficiency of fusing decisions from two models.

Datasets overview

This research makes use of a unique space dataset to address the ICIP 2021 issue of Spacecraft Recognition leveraging Knowledge of the Space Environment (SPARK) 14 , 17 , 18 . The collection contains 150,000 RGB pictures and 150,000 depth photos. This dataset was utilized to categorize 11 different types of objects, comprising 10 satellite systems.

Figure  1 shows few samples of RGB images from the SPARK dataset 14 , 17 , 18 . These samples summarize the challenges present in this dataset including random locations of objects, illuminated stars and increased contrast, a variety of orbital settings, various positions and orientations of space objects in the background, the earth having oceans and clouds in the background, a substantial noise level, and various object sizes.

Few samples of RGB images with various object sizes and backgrounds from the spark dataset 14 , 18 including AcrimSat, Aquarius, Aura, Calipso, Cloudsat, CubeSat, Debris, Jason, Sentinel-6, Terra, and TRMM in the rows 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, and 11 respectively 14 , 17 , 18 .

EfficientNet algorithm

To begin training an object identification model, images are converted into unique features that are applied to the inputs of neural networks. By utilizing CNNs to extract trainable characteristics from images, significant development has been achieved in the discipline of computer vision 19 . CNNs combine and pool picture information at several granularities, providing the model with a variety of potential configurations to focus on while learning the image identification tasks at hand.

EfficientNet is the foundation of the EfficientDet framework. EfficientNet started to investigate how CNN designs to scale 42 . There are various techniques, but it turns out that users can augment a CNN with additional parameters. Users may increase the width of each layer, the depth of the layers, or the resolution of the photos entered, or users can do a variety of these things. EfficientNet intended to develop a method for scaling CNN structures automatically 42 . The purpose of their work is to improve downstream efficiency with available free-range over depth, breadth, and resolution while remaining within the limits of target memory and FLOPs 29 .

It is the goal of feature fusion to merge samples of a particular image that are captured at various resolutions. Traditionally, the fusion employs the final several feature layers from the CNN, although the specific neural network used may differ.

EfficientDet model

Feature pyramid network (FPN) is a standard method for fusing features with a top-down direction 45 . The Path Aggregation networks (PANet) enables reverse and forward flows of feature fusion from lower to higher resolution 46 . Consequently, NAS-FPN is a feature fusion approach developed via neural architecture search (NAS) 47 . Finally, the EfficientDet model stacks these BiFPN blocks. The model scaling process alters the number of blocks.

A scaling issue was created to dynamically resize the backbone, Weighted Bi-directional FPN (BiFPN), class/box, and input image quality. The network structure scales automatically with EfficientNet-B0 to EfficientNet-B6. Thus, the amount of BiFPN stacks affects the network depth and breadth 29 . The EfficientDet framework is validated on 100,000 photos from the COCO (Common Objects in Context) dataset. Success in this area implies success in smaller particular activities. In many cases, EfficientDet outperforms other object detection methods 29 .

The authors of EfficientNet constructed the foundation model by employing a multi-objective neural network system that maximizes both efficiency and FLOP. Also, the equation they utilized is inspired by MnasNET, as seen in the Eq. ( 1 ) 48 .

\(ACC(m)\) and \(FLOPS(m)\) is expressed as the accuracy and the FLOPS of the algorithm \(m\) , \(T\) is the FLOPS’ target, and \(w =- 0.07\) is a hyperparameter that regulates the exchange among accuracy and FLOPS (floating point operations per second). Their investigation resulted in the discovery of an efficient network, which they termed EfficientNet-B0. The EfficientNet appears to be a solid foundation upon which to develop. It shows how easily scales with model performance and outperforms other CNN backbones, as demonstrated by its superior performance.

It is recommended that the BiFPN function as the feature network, where it accepts levels three to seven elements (P3,P4,P5,P6,P7) from the backbone network (EfficientNet) and implements simultaneous feature fusion top-down and bottom-up continuously 29 .

where φ = 0 for EfficientDet-D 0,1,2, … , 7 for EfficientDet-D7.

Because its level of BiFPN must be converted to tiny integers, the authors exponentially extend the width of BiFPN (#channels), as was conducted in EfficientNets, but steadily improves the depth (#layers) and it is expressed using the formula in Eq. ( 2 ). The width is maintained at the exact level of the BiFPN. However, the depth (number of layers) is raised continuously and expressed in the following equation 29 :

Considering that BiFPN employs feature levels three to seven, the input resolution has to be divisible by \({2}^{7}=128\) , which means that it linearly enhances resolutions applying the following formula 29 :

In general, an improved compound scaling approach for object recognition was presented, wherein it makes use of a simple compound coefficient, \(\phi\) , to simultaneously scale-up all features of the backbone structure, featured network, class/box network, and the input image resolution.

The EfficientDet Architecture is built on the backbone network EfficientNet 42 . Both feature network BiFPN and class/box net layers are reiterated numerous times to account for resource restrictions of varying magnitude 29 .

The proposed solution

This section discusses the proposed system of decision fusion that combines the EfficientDet model with an EfficientNet-v2 backbone to localize and classify space objects and EfficientNet-B4 model to classify the cropped images that contain space objects.

First, the experiments were conducted to train the EfficientDet object detector. In this detector, we selected EfficientNet-v2 as a backbone because it has shown superior balance between accuracy and speed in the literature. The hyperparameters were selected carefully to guarantee high performance of detection. After detector training, the evaluation metrics showed high performance in localization stage. Additionally, the detector was able to classify most of space objects with high accuracy. However, the detector was not able to classify specific category “CloudSat” of spacecraft which led to accuracy drop off. After investigation we found that testing samples of “CloudSat” category have noisy and blurred images which were not available in the training set. To address the previously mentioned problem, the cropped images that have the detected objects were augmented by adding blurring and noise. After that, these new set of training samples that contain cropped images of all categories with blurred and noisy versions of “CloudSat” samples were passed to EfficientNet-B4 CNN for training. Finally, the decisions from both models (EfficientDet and EfficientNet-B4) were fused to find the final category among eleven categories. The fusion was done by checking the prediction outcome of EfficientNet-B4 if it has “CloudSat” category, this would be the final decision. Otherwise, the final decision would be the prediction outcome of EfficientDet. Figure  2 shows the block diagram of the proposed Solution.

The images were taken from 14 , 17 , 18 .

The block diagram of the proposed Solution.

Results and discussion

Experimental setup.

For the SPARK dataset, only training and validation sets were provided with labels. Therefore, we divided the training set into two sets: 80% (72,000 images) for training, and 20% for validation (18,000 images). On the other hand, validation dataset that includes 30,000 images was used for testing. The results shown in Tables 1 , 2 , and 3 and in Figs.  3 , 4 , 5 , 6 , and 7 belong to the results of the testing dataset. The experiments conducted for this research work were done using the PyTorch and TensorFlow frameworks on an NVIDIA Tesla V100 GPU.

Confusion Matrix of EfficientDet for 11 categories of the SPARK dataset.

Confusion Matrix of EfficientDet for Debris/Satellite classification of SPARK dataset.

Confusion Matrix of EfficientNet-B4 with cropped images of SPARK dataset for 11 categories.

Confusion Matrix of the proposed solution with SPARK dataset for 11 categories.

Confusion Matrix of the proposed solution with SPARK dataset for debris/satellite classification.

The first bag of experiments was carried out for space object detection. The space images were resized to 512 × 512 before being applied to the input of the EfficientDet detector. Additionally, the images were normalized using the mean and standard deviation of the ImageNet dataset. The number of epochs was set to 10. The batch size was 4. The learning rate was 0.0002.

The second bag of experiments was carried out for space object classification using the EfficientNett-B4 CNN. The cropped images that resulted from the detector were resized to 224 × 224 before being applied to the input of the EfficientNet-B4 CNN. The layers of the base model were frozen with ImageNet weights. The last twenty layers were trained with space images. Additionally, the top layers were replaced by the following layers:

GlobalAveragePooling2D layer

BatchNormalization layer

Dropout layer with 0.2

Dense layer with 11 nodes

The hyperparameters are as follows:

learning rate of 0.0001

Optimizer of Adam

Loss function of Categorical Cross-entropy

Batch size of 64

Number of epochs of 12

Classification evaluation metrics

To evaluate the classification performance, several metrics, namely accuracy, precision, recall, F1 score, F2 score, and Perf were utilized. This section defines the performance metrics as follows:

Accuracy is a measure that calculates number of samples predicted correctly over all available samples.

Recall (Sensitivity) is a measure that calculates the proportion of actual positives that are identified correctly

Precision (positive predictive value) is a measure that calculates the proportion of positive identifications that are correct

where TP: True Positive, TN: True Negative, FP: False Positive, FN: False Negative.

F1 score is a metric that summarizes recall and precision into a single term.

F2 score is a weighted harmonic mean of precision and recall. It was used to avoid misclassification of debris as satellites.

Perf metric is a metric that is given as follows:

Detection evaluation metrics

To evaluate the detection performance, several metrics such as Intersection Over Union (IOU), mean recall, and mean precision were utilized. This section describes the performance metrics as follows:

Intersection Over Union

B: area covered by ground-truth bounding boxes.

B′: area covered by predicted bounding boxes.

IOU is an object detection metric used to measure the overlap between the actual bounding box and th predicted bounding box. A greater IoU value means a greater overlap and better detection performance. It is calculated by dividing the area of the intersection of the two boxes over the area of the union of the two boxes.

The recall in a detection task is related to the inability of an algorithm to detect objects present in the image by producing false negatives. We calculated the average recall of all classes at each IoU threshold and then calculated the mean as shown in Table 6 . Additionally, we plotted an Recall vs. IoU curve with IoU thresholds on the x-axis and recall on the y-axis. This plot illustrates the recall for each class vs. IOU thresholds  ∈  [0.5, 9.5] as shown in Fig. 10 .

The precision in a detection task is related to incorrect detection of irrelevant things in the background as an object. It can be determined by utilizing the IoU threshold. If the IoU is smaller than the threshold, it is classified as a false positive. On the other hand, if an IoU is bigger than the threshold, it is classified as a true positive. We calculated the average precision of all classes at each IoU threshold and then calculated the mean as shown in Table 6 . Additionally, we plotted Precision vs. IoU curve with IoU thresholds on the x-axis and precision on the y-axis. This plot illustrates the precision for each class vs. IOU thresholds  ∈  [0.5, 9.5] as shown in Fig. 11 . A model is considered as a good model if it has high precision and high recall.

Confidence Score

This score reflects how accurate the bounding box is and how likely there is to be an object. If no object exists, the confidence score is zero.

Experimental results

In this section, we present the results of experiments conducted to detect (localize and classify) the space objects in images of the SPARK dataset. Additionally, we evaluate the performance of the proposed solution and compare it with various baseline methods that were proposed recently in the literature. We divided the performance evaluation into two parts: classification performance evaluation and detection performance evaluation.

Classification performance evaluation

To measure classification performance, the accuracy, precision, recall, and F1 score were calculated for each class of eleven classes, and then averages were determined. The results of accuracy, precision, recall, and F1 score are shown in Tables 1 , 2 , and 3 for the following three methods:

EfficientDet with EfficientNet-v2 backbone.

EfficientNet-B4 CNN used with cropped images.

decision fusion method.

The average accuracy, precision, recall, and F1 score of EfficientDet with EfficientNet-v2 backbone were 89.5%, 84%, 82%, 79% respectively as shown in Table 1 .

Figure  3 shows the confusion matrix of the of EfficientDet with EfficientNet-v2 backbone. The high values of the elements in the main diagonal are clear. In this confusion matrix, the labels are numbered from 1 to 11 to represent the following categories: AcrimSat, Aquarius, Aura, Calipso, CloudSat, CubeSat, Debris, Jason, Sentinel-6, Terra, and TRMM, respectively. The samples with label 0 refer to the mis-detected samples. In other words, the model mis-detected 1 sample from first category, 9 samples from second category, and so on. 167 was the largest number of mis-detected samples from the “CloudSat” category.

The samples with label 5 which represent the “CloudSat” category were misclassified as labels 1, 4, 6, and 8. Only 107 out of 2500 samples were classified correctly. The reason was that images in the “Cloudsat” category during the initial testing set were noisy and blurry and were different from the images in the training set.

The confusion matrix of the binary debris/satellite classification task that used the EfficientDet model is shown in Fig.  4 . The matrix is evidence of the high capability of the classifier to identify debris out from other categories.

The average accuracy, precision, recall, and F1 score of EfficientNet-B4 with cropped images were 91.54%, 86%, 84%, 84%, respectively as shown in Table 2 .

Figure  5 shows the confusion matrix of the EfficientNet-B4 model with cropped images. The high values of the elements in the main diagonal are clear. The number of samples with label 5, which represents “CloudSat” category, has been increased remarkably compared to the previous EfficientDet model. In other words, 1334 out of 2500 samples were classified correctly. The reason was that we augmented images with “Cloudsat” category in the training set by adding blurring and noise and then passed the set into the EfficientNet-B4 model for training.

The average accuracy, precision, recall, and F1 score of the proposed solution of decision fusion were 93.98%, 87%, 86%, 86%, respectively as shown in Table 3 .

Figure  6 shows the confusion matrix of the of the proposed solution of decision fusion. The high values of the elements in the main diagonal are evident. The number of samples with label 5 which represents the “CloudSat” category has been increased compared to the previous EfficientNet-B4 CNN. In other words, 1378 out of 2500 samples were classified correctly. The reason was that we combined the decisions from two previously mentioned models that include EfficientDet with an EfficientNet-v2 backbone and EfficientNet-B4 CNN.

The confusion matrix of binary debris/satellite classification of the proposed solution of decision fusion is shown in Fig.  7 . The matrix shows the high capability of the classifier to identify debris out from other categories.

Ablation study

In this section, an ablation study is described to validate the significance of decision fusion that made the final decision by combining decisions from EfficientDet with an EfficientNet-v2 backbone and EfficientNet-B4 CNN. The proposed solution was compared with the baseline methods in terms of accuracy, F2-score, and Perf metric as shown in Table 4 . The accuracy here is related to only 10 categories of satellites and ignore the “debris” category that the F2 score focuses on.

It was found that decision fusion was able to make significant improvements in classification over EfficientDet by increasing the accuracy from 88.53 to 93.46% and the performance metric from 1.8727 to 1.9223.

The proposed solution for decision fusion that combines decisions from EfficientDet with an EfficientNet-v2 backbone and EfficientNet-B4 CNN was compared with the baseline method in terms of accuracy, F2-score, and Perf metric as shown in Table 4 . The baseline method is the multimodal CNNs 2 that includes a pre-trained ResNet50 CNN connected to a support vector machine (SVM) classifier for classification of RGB images and an end-to-end CNN for classification of depth images. It was found that the proposed solution was able to make significant improvements in classification by increasing accuracy from 86.77 to 93.46%, F2 score from 95.39 to 98.77%, and performance metric from 1.8216 to 1.9223.

In Table 5 , the proposed solution was also compared with the baseline methods in the literature in terms of accuracy. The accuracy here refers to the average accuracy of all 11 categories including satellites and debris. In 2 , the authors used ResNet50 CNN + SVM with cropped RGB images only, just as our proposed method does, and yielded 85% accuracy. Then, they proposed multimodal CNNs using both RGB and depth images after detection and cropping to increase accuracy from 85 to 89% as shown in Table 5 . Additionally, AlDahoul et al. 43 proposed various methods to recognize spacecrafts as shown in Table 5 . Some methods utilized only RGB images, just as ours does. A vision transformer was utilized with whole RGB images without detection and yielded 81% accuracy. On the other hand, some methods in 43 used both RGB images and depth images to improve the recognition accuracy. Both EfficientNetB7-End2End CNN and Vision Transformer-End2End CNN have accuracies of 85% using also whole images without detection.

It is obvious in Table 5 that our proposed methods can outperform other methods in terms of accuracy, utilizing only RGB images to produce 89.5% with EfficientDet alone, 91.5% with EfficientNet-B4 alone, and 94% with decision fusion that combines decisions from EfficientDet with an EfficientNet-v2 backbone and EfficientNet-B4 CNN.

Detection performance evaluation

To evaluate the detection model, a distribution of IoUs between ground truth bounding boxes and predicted bounding boxes was plotted in Fig.  8 . It highlights the fact that IOUs have high values which are above 0.8. Additionally, 282 images out of 30,000 images were mis-detected during the detection stage. Furthermore, a distribution of confidence scores was plotted in Fig.  9 . It is obvious that the detector has high confidence scores.

Distribution of IOU in EfficientDet model.

Distribution of Confidence scores in EfficientDet model.

An object detection method is evaluated by calculating the detection precision and recall. In other words, the detector is considered optimal if it has high precision and high recall.

The inability of a detection algorithm to detect objects present in the image by producing false negatives led to lower recall. In Table 6 , the average recall of all classes at each IoU threshold was calculated and then the mean was determined. Additionally, we plotted recall against IoU on a curve with IoU thresholds on the x-axis and recall on the y-axis as shown in Fig.  10 . This plot illustrates the recall for each class vs. IOU thresholds  ∈  [0.5, 9.5].

Recall vs. IOUs for 11 classes using the proposed solution.

The wrong detection of irrelevant things in the background and labelling them with wrong object labels led to lower precision. Furthermore, smaller values of IoUs than the predefined threshold yield lower precision. In Table 6 , the average precision of all classes at each IoU threshold was calculated and then the mean was found. Additionally, we plotted precision against IoU on a curve with IoU thresholds on the x-axis and precision on the y-axis as shown in Fig.  11 . This plot illustrates the precision for each class vs. IOU thresholds  ∈  [0.5, 9.5].

Precision vs IOUs for 11 classes using the proposed solution.

The limitation in EfficientDet with an EfficientNet-v2 backbone that was trained on the original training images was its inability to recognize images from the Cloudsat category well because of noisy and blurry images from this specific category in the testing set. Therefore, the Cloudsat category was misclassified and predicted wrongly with 107 correct predictions over 2500 images with an average accuracy of 89.5% for 11 categories. To address this problem, EfficientNet-B4 CNN was trained on cropped images after augmenting the images from the CloudSat category by blurring and adding Gaussian noise. As a result, the number of correct images from the CloudSat category was increased from 107 to 1334 images with an average accuracy of 91.54% for 11 categories.

Finally, the decision fusion approach was applied to combine decisions from both models—EfficientDet with an EfficientNet-v2 backbone and EfficientNet-B4 CNN. The final decision of final category was found by fusing two decisions and was found to increase the number of correct images from the CloudSat category to 1378 images with an average accuracy of 94% for 11 categories.

Figures  12 , 13 , and 14 illustrate a few samples to show the overlap between actual bounding box (red) and predicted box (blue) for the Sentine, TRMM, and Terra categories. It is clear that EfficientDet was able to predict boxes that have large agreements with the ground truth boxes even if the backgrounds were complex as shown in the figures. Furthermore, the ability of the detector to localize small size objects belonging to various categories was behind the significant improvement in accuracy and performance metrics compared to existing methods. Even if several challenges were present in the dataset including random locations of objects, illuminated scenes and increased contrast, a variety of orbital settings, various positions and orientations of space objects in the background, and a substantial noise level, the detector was able to localize and classify space objects with favourable classification metrics: accuracy (94%), performance (1.9223); and detection metrics: mean precision (78.45%) and mean recall (92.00%).

Few examples to show the overlap between actual bounding box (red) and predicted bounding box (blue) for the Sentine category 14 , 17 , 18 .

Few examples to show the overlap between actual bounding box (red) and predicted bounding box (blue) for the TRMM category 14 , 17 , 18 .

Few examples to show the overlap between actual bounding box (red) and predicted bounding box (blue) for the Terra category 14 , 17 , 18 .

The advantages of the proposed solution are that:

The task was formulated as an image detection problem. It can localize space objects by predicting the four coordinates of the box surrounding the spacecraft and debris. Additionally, it can classify the cropped images that contain space object into 11 categories. In other words, the proposed solution can focus attention on regions of interest (ROIs) that contain space objects inside the image and ignore irrelevant objects in the background. This matter plays a significant role in improving recognition accuracy.

The proposed solution can perform well in space missions because it is robust against all challenges present in this dataset including random locations of objects, illuminated stars and increased contrast, a variety of orbital settings, various positions and orientations of space objects in the background, the earth having oceans and clouds in the background, a substantial noise level, and various object sizes.

RGB images are enough to be used for the space object detection method. Therefore, there is no need for depth images that other methods utilized.

Conclusion and future work

The study presented in this paper contributes to attract the research community by highlighting an interesting new challenge that enriches the body of knowledge. It proposed an efficient solution to localize and recognize space objects such as spacecraft and debris to enhance the performance of SSA system. In this research work, EfficientDet with an EfficientNet-v2 backbone was trained on the SPARK dataset to localize space objects in RGB images by predicting four coordinates of the boxes surrounding the objects. Additionally, a multi-modal learning approach is proposed for spacecraft classification using only RGB images to combine decisions from EfficientNet-v2 and EfficientNet-B4 that were trained on the SPARK dataset. The fused decision block was added to make the final decision about object class. We evaluated the proposed solution using various metrics for classification such as accuracy, and F1 score and for detection such as IOU, mean recall, and mean precision. Furthermore, we compared the proposed solution with other methods that utilized the same dataset.

An ablation study was done to validate the significant improvement in classification accuracy by using multi-modal learning which creates the final decision by combining decisions from efficientNet-v2 and EfficientNet-B4 CNNs. It was found that the proposed combination of EfficientDet with an EfficientNet-v2 backbone and EfficientNet-B4 CNN was able to outperform state of the art methods in terms of accuracy (94%), and performance metric (1.9223%) for object classification; and in terms of mean Precision (78.45%) and mean recall (92.00%%) for object detection. This study achieved its goal to enhance classification performance largely by focusing attention on regions of space objects and by ignoring other irrelevant objects in the backgrounds. Therefore, the proposed method of space object detector is a good feasible solution that can be utilized in real task of SSA system.

In the future, we plan to improve the performance of the solution by training recent object detectors such as YOLOv5 to evaluate its ability to detect space objects in this challenging dataset. Furthermore, to implement this solution on edge computing for real missions of SSA systems, we plan to train recent light versions of object detectors such as YOLOv5n 49 and nanoDet 50 using the SPARK dataset to balance between accuracy and inference speed.

Data availability

The Dataset belongs to University of Luxembourg and LMO. You may contact prof. Djamila Aouada ([email protected]) to request this dataset for research purposes.

ESA—Space Situational Awareness—SSA (accessed 9 Jan 2022); https://www.esa.int/About_Us/ESAC/Space_Situational_Awareness_-_SSA .

AlDahoul, N., Karim, H. A. & Momo, M. A. RGB-D based multimodal convolutional neural networks for spacecraft recognition. In 2021 IEEE International Conference on Image Processing Challenges (ICIPC) 1–5. https://doi.org/10.1109/ICIPC53495.2021.9620192 (2021).

Sharma, S., Beierle, C. & D’Amico, S. Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks. In 2018 IEEE Aerospace Conference 1–12. https://doi.org/10.1109/AERO.2018.8396425 (2018).

Ke, L. & Quanxin, W. Study on signal recognition and diagnosis for spacecraft based on deep learning method. In 2015 Prognostics and System Health Management Conference (PHM) 1–5. https://doi.org/10.1109/PHM.2015.7380040 (2015).

García, A. M. et al . LSPnet: A 2D localization-oriented spacecraft pose estimation neural network. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 2048–2056. https://doi.org/10.1109/CVPRW53098.2021.00233 (2021).

Proenca, P. F. & Gao, Y. Deep learning for spacecraft pose estimation from photorealistic rendering. In 2020 IEEE International Conference on Robotics and Automation (ICRA) 6007–6013. https://doi.org/10.1109/ICRA40945.2020.9197244 (2020).

Phisannupawong, T. et al. Vision-Based Spacecraft Pose Estimation via a Deep Convolutional Neural Network for Noncooperative Docking Operations . https://doi.org/10.3390/aerospace7090126 (2020).

Kim, J., Lee, J. K. & Lee, K. M. Accurate image super-resolution using very deep convolutional networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 1646–1654. https://doi.org/10.1109/CVPR.2016.182 (2016).

Selvaraju, R. R. et al . Grad-CAM: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV) 618–626. https://doi.org/10.1109/ICCV.2017.74 (2017).

He, K, Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778. https://doi.org/10.1109/CVPR.2016.90 (2016).

Qiu, L., Tang, L. & Zhong, R. Toward the recognition of spacecraft feature components: A new benchmark and a new model. Astrodynamics https://doi.org/10.1007/s42064-021-0103-3 (2021).

Article   Google Scholar  

Yan, Z., Song, X. & Zhong, H. Spacecraft detection based on deep convolutional neural network. In 2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP) 148–153. https://doi.org/10.1109/SIPROCESS.2018.8600520 (2018).

Chabot, T. et al . Vision-based navigation experiment onboard the removedebris mission. In GNC 2017–10th International ESA Conference on Guidance, Navigation & Control Systems 1–23 (2017).

Musallam, M. A. et al . SPARK: SPAcecraft Recognition leveraging Knowledge of Space Environment. CoRR , vol. abs/2104.05978. https://arxiv.org/abs/2104.05978 (2021).

Forshaw, J. L. et al. RemoveDEBRIS: An in-orbit active debris removal demonstration mission. Acta Astronaut. 127 , 448–463. https://doi.org/10.1016/j.actaastro.2016.06.018 (2016).

Article   ADS   Google Scholar  

Opromolla, R., Fasano, G., Rufino, G. & Grassi, M. Uncooperative pose estimation with a LIDAR-based system. Acta Astronaut. 110 , 287–297. https://doi.org/10.1016/j.actaastro.2014.11.003 (2015).

Musallam, M. A. et al. Spacecraft recognition leveraging knowledge of space environment: simulator, dataset, competition design and analysis. In 2021 IEEE International Conference on Image Processing Challenges (ICIPC) 11–15. https://doi.org/10.1109/ICIPC53495.2021.9620184 (2021).

SPARK Challenge. https://2021.ieeeicip.org/ChallengeSessions.asp (2022).

Lecun, Y., Bottou, L., Bengio, Y. & Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 86 (11), 2278–2324. https://doi.org/10.1109/5.726791 (1998).

Girshick, R., Donahue, J., Darrell, T. & Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Conf. Comput. Vis. Pattern Recogn. 2014 , 580–587. https://doi.org/10.1109/CVPR.2014.81 (2014).

Ren, S., He, K., Girshick, R. & Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39 (06), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031 (2017).

Girshick, R., Donahue, J., Darrell, T. & Malik, J. Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 38 (1), 142–158. https://doi.org/10.1109/TPAMI.2015.2437384 (2016).

He, K., Gkioxari, G., Dollár, P. & Girshick, R. Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42 (2), 386–397. https://doi.org/10.1109/TPAMI.2018.2844175 (2020).

Liu, W. et al. SSD: Single shot multibox detector. In Computer Vision—ECCV 2016 , Cham 21–37. https://doi.org/10.1007/978-3-319-46448-0_2 (2016).

Chiu, Y.-C., Tsai, C.-Y., Ruan, M.-D., Shen, G.-Y. & Lee, T.-T. Mobilenet-SSDv2: An improved object detection model for embedded systems. Int. Conf. Syst. Sci. Eng. 2020 , 1–5. https://doi.org/10.1109/ICSSE50014.2020.9219319 (2020).

Redmon, J. & Farhadi, A. YOLOv3: An Incremental Improvement. CoRR , vol. abs/1804.02767. http://arxiv.org/abs/1804.02767 (2018).

Bochkovskiy, A.,Wang, C.-Y. & Liao, H.-Y. M. YOLOv4: Optimal speed and accuracy of object detection. CoRR , vol. abs/2004.10934. https://arxiv.org/abs/2004.10934 (2020).

Zhu, X., Lyu, S., Wang, X. & Zhao, Q. TPH-YOLOv5: Improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios. CoRR , vol. abs/2108.11539. https://arxiv.org/abs/2108.11539 (2021).

Tan, M., Pang, R. & Le, Q. V.“EfficientDet: Scalable and efficient object detection. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10778–10787. https://doi.org/10.1109/cvpr42600.2020.01079 (2020).

Strube, M. J. et al . Raven: An On-Orbit Relative Navigation Demonstration Using International Space Station Visiting Vehicles (2015).

Yol, A., Marchand, E., Chaumette, F., Kanani, K. & Chabot, T. Vision-Based Navigation in Low Earth Orbit Int. Symp. on Artificial Intelligence, Robotics and Automation in Space, i-SAIRAS'16, Jun 2016, Beijing, China (2016).

Naasz, B. J. et al. The HST SM4 relative navigation sensor system: overview and preliminary testing results from the flight robotics lab. J. Astronaut. Sci. 57 (1), 457–483. https://doi.org/10.1007/BF03321512 (2009).

Du, X., Liang, B., Xu, W. & Qiu, Y. Pose measurement of large non-cooperative satellite based on collaborative cameras. Acta Astronaut. 68 (11–12), 2047–2065. https://doi.org/10.1016/J.ACTAASTRO.2010.10.021 (2011).

Shi, J.-F., Ulrich, S. & Ruel, S. Spacecraft pose estimation using principal component analysis and a monocular camera. In AIAA Guidance, Navigation, and Control Conference 1034. https://doi.org/10.2514/6.2017-1034 (2017).

Sharma, S. & D’Amico, S. Neural network-based pose estimation for noncooperative spacecraft rendezvous. IEEE Trans. Aerosp. Electron. Syst. 56 (6), 4638–4658. https://doi.org/10.1109/TAES.2020.2999148 (2020).

Szegedy, C. et al. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 1–9 https://doi.org/10.1109/CVPR.2015.7298594 (2015).

Simonyan, K. & Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition . arXiv:1409.1556 (2014).

Sonawani, S. et al . Assistive Relative Pose Estimation for On-Orbit Assembly Using Convolutional Neural Networks . arXiv:2001.10673 (2020).

Kisantal, M. et al. Satellite pose estimation challenge: Dataset, competition design, and results. IEEE Trans. Aerosp. Electron. Syst. 56 (5), 4083–4098. https://doi.org/10.1109/TAES.2020.2989063 (2020).

Pose Estimation Challenge (accessed 3 February 2022); https://kelvins.esa.int/satellite-pose-estimation-challenge/ .

Unity real-time development platform|3D, 2D VR & AR Engine (accessed 3 February 2022); https://unity.com/ .

Tan, M. & Le, Q. V. EfficientNet: Rethinking Model scaling for convolutional neural networks. CoRR , vol. abs/1905.11946. http://arxiv.org/abs/1905.11946 (2019).

AlDahoul, N., Karim, H. A. & Momo, M. A. RGB-D based multi-modal deep learning for spacecraft and debris recognition. Sci. Rep. 12 , 3924. https://doi.org/10.1038/s41598-022-07846-5 (2022).

Article   ADS   CAS   Google Scholar  

Hor, S. L. et al. An evaluation of state-of-the-art object detectors for pornography detection. IEEE Int. Conf. Signal Image Process. Appl. 2021 , 191–196. https://doi.org/10.1109/ICSIPA52582.2021.9576796 (2021).

Lin, T.-Y. et al . Feature pyramid networks for object detection. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 936–944. https://doi.org/10.1109/CVPR.2017.106 (2017).

Liu, S., Qi, L., Qin,H., Shi, J. & Jia, J. Path aggregation network for instance segmentation. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 8759–8768. https://doi.org/10.1109/CVPR.2018.00913 (2018).

Ghiasi, G., Lin, T.-Y., Pang, R. & Le, Q. V. NAS-FPN: Learning scalable feature pyramid architecture for object detection. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 7029–7038. https://doi.org/10.1109/CVPR.2019.00720 (2019).

Tan, M., Chen, B., Pang, R., Vasudevan, V. & Le,Q. V. MnasNet: Platform-aware neural architecture search for mobile. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2815–2823. https://doi.org/10.1109/CVPR.2019.00293 (2019).

Delong, Q., Weijun, T. Qi, Y. & Jingfeng, L. YOLO5Face: Why Reinventing a Face Detector arXiv:2105.12931 (2022).

RangiLyu, NanoDet. https://www.github.com/RangiLyu/nanodet (2021).

Download references

Acknowledgements

The SPARK dataset used in this work was proposed in an ICIP2021 challenge. Many thanks to the University of Luxembourg and LMO for sharing their dataset.

This research project was funded by Multimedia University, Malaysia.

Author information

Authors and affiliations.

Faculty of Engineering, Multimedia University, Cyberjaya, Malaysia

Nouar AlDahoul & Hezerul Abdul Karim

College of Engineering and Technology, University of St. La Salle, 6100, Bacolod, Philippines

Angelo De Castro & Myles Joshua Toledo Tan

New York University Abu Dhabi, Abu Dhabi, UAE

Nouar AlDahoul

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization by N.A.; data curation by N.A.; formal analysis by N.A., M.J.T.T., H.A.K.; funding acquisition by H.A.K.; investigation by N.A.; methodology by N.A.; project administration by H.A.K.; software by N.A.; validation by N.A.; visualization by N.A.; writing—original draft preparation by N.A., A.D.C.; writing—review &editing by N.A., M.J.T.T, H.A.K.

Corresponding author

Correspondence to Nouar AlDahoul .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

AlDahoul, N., Karim, H.A., De Castro, A. et al. Localization and classification of space objects using EfficientDet detector for space situational awareness. Sci Rep 12 , 21896 (2022). https://doi.org/10.1038/s41598-022-25859-y

Download citation

Received : 23 July 2022

Accepted : 06 December 2022

Published : 19 December 2022

DOI : https://doi.org/10.1038/s41598-022-25859-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Computer vision tasks for intelligent aerospace perception: an overview.

• HuiLin Chen

Science China Technological Sciences (2024)

Review of Machine-Learning Approaches for Object and Component Detection in Space Electro-optical Satellites

• Kebei Zhang

International Journal of Aeronautical and Space Sciences (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Biomimetic space debris removal: conceptual design of bio-inspired active debris removal scenarios

• Original Paper
• Open access
• Published: 22 April 2022
• Volume 15 , pages 237–252, ( 2023 )

Cite this article

You have full access to this open access article

• E. Banken 1 ,
• V. E. Schneider 1 ,
• M. K. Ben-Larbi 2 , 4 ,
• L. Pambaguian 3 &
• J. Oeffner   ORCID: orcid.org/0000-0002-3314-7557 1  

4790 Accesses

5 Citations

Explore all metrics

The ever-increasing number of man-made space debris creates the need for new technologies to mitigate it. Therefore, within the ESA-funded project BIOINSPACED, biologically inspired solutions for active debris removal were investigated, conceptualized and integrated to innovative and comprehensive scenarios. In the following, the collection process of existing and new biomimetic concepts as well as the evaluation of ten concepts based on a feasibility analysis will be presented. Out of the ten, the three most promising scenarios, were chosen for further investigation and further elaborated in detail specifying the biological models incorporated as well as how the scenario could be implemented in a simple demonstrator. The first scenario (A) is a gecko kit canon and describes a system that fires deorbiting kits towards the target from a safe distance. The second scenario (B) involves a robotic arm with a gecko-adhesive end-effector and a bee-inspired harpoon to achieve a preliminary and subsequent rigid connection to the target. The last scenario (C) is mimicking a Venus Flytrap and its bi-stale mechanism to capture its prey. One of these scenarios will be manufactured and built into a demonstrator to showcase biology’s potential for the development, optimization and improvement of technologies, especially within the space industry.

Similar content being viewed by others

Synthetic Muscle™ for Deep Space Travel and Other Applications on Earth and in Space

Review on Bioinspired Planetary Regolith-Burrowing Robots

Progress in the development of small-celestial-body anchoring robots

Avoid common mistakes on your manuscript.

1 Introduction

The increasing utilization of the extraterrestrial environment is associated with a rising number of satellites, spacecrafts and devices occupying the orbits around Earth [ 1 , 2 ]. In 2020, only about 10% of the approximately 28000 trackable objects in space were active satellites. Thus, the majority of these objects is space debris, which are man-made objects without functional use, such as retired payloads, spent upper stage rocket bodies, and fragments from collisions and explosions. However, the growing population of debris objects orbiting Earth poses a serious risk to current and future missions. Collision processes can cause a chain reaction and render entire orbital regions unusable. The highest risk is associated with large inactive objects, which contribute 99% to the environmental index, a metric used by ESA to measure the risk of an object within the space environment [ 3 ]. It has, therefore, become apparent that reliable and affordable systems for non-functional object removal or servicing is essential to guarantee safe and sustainable access to orbits around Earth. This evolution has led to the formation of a novel research field investigating active space debris removal (ADR) with an increasing number of activities. However, many of the existing ADR concepts still remain in the developmental stage, require proof-of-concept efforts or real scenario field testing.

Biomimetics and its application to find innovative technical solutions has proven beneficial throughout many industries and bio-inspired products, such as lotus paint (Lotusan), Gecko-tape, sharkskin inspired swimsuits, and Velcro [ 4 , 5 , 6 , 7 , 8 ] have been successfully established on the market. Thus, its potential is more and more recognized and increasingly considered for product design, development and optimization. For some time now, biological organisms have also served as an inspiration for technical development in aerospace engineering and space exploration. For example, the wood wasp drilling into the bark of trees with its ovipositor has been used as a model for surgical instruments on Earth, but has also been considered as a solution for extraterrestrial drilling and sampling for decades [ 9 , 10 , 11 ]. An X-ray telescope with lobster eye optics presents another, more recently developed biomimetic technology to discover remote objects in space outside Earth’s atmosphere and was used on the Czech nanosatellite launched in 2017 [ 12 ]. Some of the recently established ADR concepts already include biologically inspired ideas such as the prominent example of using the gecko’s feet as a model for adhesive materials implemented in a gripper to allow for docking to debris in space without requiring a specific adapter or compliant object [ 13 , 14 , 15 ].

Therefore, looking at biology, its great diversity of mechanisms and its evolved features often reveals transferable concepts and may provide valuable contributions to ADR. Nature presents an abundance of features that have evolved to fit certain environmental requirements or cope with external pressures. Thus, multiple approaches exist to fulfil similar tasks and activities, which present many qualities also essential for space systems, such as response-stimuli adaptability, robustness and lightweight construction, autonomy and intelligence, energy efficiency, and self-repair or healing capabilities [ 16 , 17 ]. Hence, biological mechanisms can be transferred and adapted to improve or even revolutionize traditional engineering approaches.

2 The BIOINSPACED project

BIOINSPACED is an acronym that stands for bioinspired solutions for space debris removal. The project was funded by the European Space Agency (ESA) and is conducted by Fraunhofer CML with Technical University of Braunschweig (TUBS) as a subcontractor. It commenced in June 2020 with the goal to contribute to ESA’s CleanSpace initiative by developing biomimetic solutions for innovative technologies to support the removal of space debris, especially in low earth orbit (LEO).

During the project’s initial phase, the elementary steps for ADR were analyzed, identifying the phases included in such a mission as well as related requirements. Afterwards, nature’s pool of existing concepts and possibly new biological mechanisms were reviewed with the prospects of finding ones with potential for ADR. All of these concepts were collected within the BIOINSPACED catalogue, a comprehensive and interactive database, and evaluated using a feasibility analysis (see Sect.  3 ). The best performing and thus most promising concepts were integrated into several holistic mission scenarios. After a collaborative discussion among Fraunhofer CML, TUBS and ESA, three of these most promising scenarios were selected for further investigation and conceptual design, in preparation of choosing one to be build into a demonstrator and undergo preliminary experiments. BIOINSPACED aims to not only present the diversity of biological examples that hold potential for implementation within ADR missions, but to demonstrate the bio-inspired concepts within ESA and showcase the potential of biomimetics for the space industry in general.

3 BIOINSPACED catalogue and feasibility analysis

To collect existing biomimetic and new biological concepts suitable for processes involved in ADR, three different approaches were applied: First, a thorough literature review was conducted, studying existing biomimetic models, prototypes and products within the fields of robotics, materials science, kinematics, and space technology among others. Then, nature’s pool of ideas was screened by browsing biological research papers, nature documentaries and flipping through other materials to propose new solutions, which include those demonstrating great challenges for “traditional engineering”. Finally, three biomimetic brainstorming workshops were held with participants from the fields of space industry, biology and biomimetics, resulting in a great amount of new biological principles and mechanisms as well as intensive discussions of their suitability for space application. More information on the collection process and the three approaches can be found in Banken et al. [ 18 ].

All of the collected concepts were added to the BIOINSPACED catalogue, a comprehensive and informative database with rated information on several types of biological and biomimetic systems. It also provides an interactive and customizable tool for accessing and utilizing available information according to user needs and summarizes biology’s potential for its application in space engineering. While the presented catalogue was constructed within the scope of the BIOINSPACED project and its predefined requirements, it can also be utilized in the future for finding biomimetic solutions that prove beneficial in different space contexts. The complete and detailed catalogue of biomimetic concepts is presented in the Supplementary Material. Fraunhofer CML can be approached for further information.

Based on the established BIOINSPACED catalogue, a feasibility analysis was conducted to evaluate the importance and relevance of collected concepts. This analysis was based on the four parameters that were calculated into an overall score called ‘BIOINSPACED Applicability Score’ (BAS). :

Technical feasibility (TF): presents a basic indicator for a concept’s functionality and evaluates the overall idea concerning the possibility of its implementation into a technical system, especially with regard to the final goal of building a demonstrator and associated design as well as manufacturing constraints.

Biomimetic applicability (BA): analyses a biological model and indicates its potential to be adapted and transferred into a technical system

Space applicability (SA): assesses the possibility of the model’s implementation and employment within the space environment. This, too, includes considerations of the predicted concept reliability.

Novelty factor (NF): examines the originality of concepts and investigates currently available ideas in literature. This factor is heavily influenced by the amount and type of literature published, discussing either a concept’s aerospace application, any kind of industry application, its mere biological functioning or none whatsoever.

The parameters were evaluated by six scientists from Fraunhofer CML and two from TUBS individually by assigning a score from 1 to 6 (1 indicated the best ranking and assumed performance) to each parameter for every concept. These ratings were summarized, averaged for the individual parameters and then multiplied by the weighting factor as indicated in Eq.  1 .

While TF and BA were assigned the highest weighting factors as they determine whether a technology can be established based on the pre-defined biological model at all, the project was aiming for technical solutions within the space industry. Thus, the SA factor was assigned an only slightly lower weighting factor. Finally, another project requirement was the development of a demonstrator showcasing new and innovative biomimetic concepts, therefore, making the novelty an important requirement within the scope of the project, yet less important than the overall development and implementation potential. It is important to stress that the results are dependent on the chosen weighting factors. Several evaluators from Fraunhofer CML and TUBS with extensive backgrounds in the different fields of biomimetics, aerospace and mechanical engineering were included and participated in discussions about the weighting factors, and provided input based on their respective expertise. Nevertheless, the assessment of the individual factors is ultimately based on subjective ratings, thus, a different group of evaluators may draw different conclusions. Within the scope of this project and in consultation with the ESA project officer, the selected weighting factors were considered sufficient to rate the biological concepts.

Summing up those four parameter scores resulted in the overall BAS for each concept. Those ranked concepts were then grouped into overlying working principles to provide a better overview of available mechanisms that may aid the process of an ADR mission. The best performing 24 grouped principles were then presented and discussed by project partners and ESA employees, collaboratively deciding on 10 principles to be further investigated and integrated into holistic scenarios. The selected principles and a short description of their functioning are summarized in Banken et al. [ 18 ]:

Equation 1: Formula to calculate the overall score for ‘BIOINSPACED applicability score’ (BAS) including all four parameters evaluated by CML and TUBS and the respective weighting factor. TF: technical feasibility, BA: biomimetic applicability, SA: space applicability, NF: novelty factor.

4 ADR environment and mission requirements

4.1 the adapted adr ecosystem.

The conventional phases associated with rendezvous missions are detailed by Fehse [ 19 ], covering the launch, phasing, far- and close range rendezvous with the target, capture and finally the removal. Aiming to integrate the established principle solutions into holistic ADR scenarios demanded for an adaptation of these steps. Since launch, phasing and far-range rendezvous remain very similar to common rendezvous missions (with cooperative targets, such as the ISS), they were excluded from the ADR ecosystem of this study to focus available resources on finding solutions for the other phases. In addition, sufficient information during the execution of the first three ADR phases can be obtained from ground-based systems, such as radar and passive optical telescopes, providing a great amount of data on debris detection, tracking and identification [ 20 ]. In addition, the abundance of biological models for the remaining phases was assumed to be much greater, which was supported by the quality and quantity of concepts collected within the catalogue.

The ADR ecosystem was, however, extended by other steps in case the established scenarios demanded additional actions to guarantee a successful removal mission as suggested by for example Maediger and colleagues [ 21 ], and possibly included an inspection flyby, detumbling actions or a pre-attachment as per requirement. During the inspection flyby phase, the chaser uses its on-board detection and sensing systems continuously pointing at the target while traveling around it at a constant distance on the expense of spending additional fuel. This allows the procurement of an adequate amount of information especially related to the integrity of the target and its rotational motion [ 21 ]. Detumbling actions can be applied before or after the capture or physical contact between the chaser and the target, ranging from plume impingement [ 22 ] or magnetic torque generation [ 23 , 24 ], to the use of a kinematically redundant robotic arm [ 25 , 26 ] or a brush-type contractor [ 27 ]. Those actions are necessary if the rotational velocities of the target exceed servicing capabilities and, therefore, prohibit a safe approach for the attachment and capture [ 28 ].

In terms of permanent connections formed between the chaser and the target, nets and harpoons are prominent capture concepts, and among the only ones demonstrated on-orbit during the ’RemoveDebris’ mission in 2019 [ 29 , 30 , 31 ]. While this mission displays a huge leap in space debris removal and tests showed promising results, harpoons are still associated with high risks of additional debris production due to the forces required to penetrate the target’s surface material while preventing a large impulse generation and thus pushing the target from its current course. Furthermore, complex rope dynamics between chaser and target have not been investigated in this mission and present a technical challenge. Therefore, the last additional phase included within the ADR ecosystem is a preliminary attachment that enables a safe but less rigid connection with the target first that does not generate high impulse forces, prohibits the target’s escape, and facilitates a subsequent rigid connection using more complex and high-energy approaches.

The adapted ADR ecosystem defined within the scope of the BIOINSPACED project as indicated in Fig.  1 is used to specify and visualize the complete process of space debris removal, where a holistic point of view on the composition of the overall mission is delivered.

Representation of the adapted ADR Ecosystem. The phases outside the green rectangle as well as the ones highlighted in white inside the rectangle present the conventional ADR phases identified by the BIOINSPACED project. These phases were later adapted to include the ones specifically tailored to biomimetic ADR scenarios (highlighted in light green) and excludes the ones not exclusively applicable to ADR. Thus, within the scope of the project, only the phases inside the green frame represent the ecosystem for bio-inspired ADR scenarios

4.2 From concepts to mission scenarios

Over the course of this project, concepts and their application within biomimetic ADR were adapted and re-defined as the project progressed. Therefore, Fig.  2 presents the conversion from individual concepts over principles into scenarios and the final demonstrator. As described in Sect.  3 , the 130 individual existing and new biomimetic concepts were collected using literature review, brainstorming activities and several biomimetics workshops. These concepts, each covering a single feature of one or several species of an organisms (demonstrating similar features) were grouped into overlying principles to simplify their evaluation and selection. These principles described the same function, for example ’adhesion’, ’penetration’ or ’folding’ without using the same process, feature or mechanism. Out of these principles, ten were selected during the ’Design Space Review’ meeting to be integrated into ten holistic scenarios, offering solutions for the ADR phases defined in subsection  4.1 . In the following subsection  4.3 and  4.4 , the process of integration, development and assessment of the ten scenarios will be described in great detail. Finally, the three scenarios selected for further detailed investigation in regard to the final demonstrator were decided upon during the ’Debris Removal Selection Review’ meeting. These scenarios will be presented in Sect.  5 .

During the ’Final Scenario Selection Review’ meeting, one of these three scenarios will be chosen for further development and ultimately be transferred into a preliminary functioning demonstrator. This demonstrator will display the incorporated biomimetic concepts and indicate how they can be transferred into basic but working technical solutions. This demonstrator can then be used by ESA to show the beneficial role biomimetics can play in the development of aerospace technologies and will be presented at the upcoming International Conference on Advanced Manufacturing (ICAM22), displaying the manufacturing and integration of the most promising concepts.

Transformation flow of collected concepts over principles to scenarios. It demonstrates the selection process over the course of the project and the conversion of collected concepts in the beginning into overlying working principles. These principles were then combined and integrated to form ten scenarios out of which three were chosen for further investigation. In the final task of this project, one of these scenarios will be built into a demonstrator. The individual task decisions refer to the milestone meetings: DSR design space review, DRSR debris removal selection review, FSSR final scenario selection review

4.3 Building biomimetic ADR scenarios

With the help of individual Zwicky boxes for each principle, which are a favoured tool for structuring and investigating complex problems with multiple solution approaches and are, thus, often used as means for analysis in biomimetics [ 32 , 33 ], all possible solutions for the implementation of each principle were established. Afterwards, the three most promising principle solutions were determined and used for their integration and combination with the solutions of the remaining principles into ten holistic ADR scenarios. This resulted in the inclusion of not only chosen principle mechanisms but also concepts from the BIOINSPACED catalogue that demonstrated beneficial improvements over conventional mission technologies. Therefore, the number of concepts contained within one scenario reached up to the maximum of 13 different ones.

Subsequently, the ten scenarios were evaluated using a trade-off analysis to evaluate the feasibility of each regarding their implementation potential into a demonstrator under consideration of the following mission critical parameters:

Technical feasibility: in this analysis referred to as ’ T ’ to demonstrate the difference to the prior technical feasibility factor used before in Sect.  3 : potential of implementing a scenario into a technical system (no. of moving parts, time critical activities)

Technical complexity ( Tc ): intricacy of the system (component interactions, motion control requirements)

Engineering effort ( E ): technological readiness level (Use of existing materials/ devices, environment appropriate)

Energy requirements ( Er ): requirements for motion and course control required (movability of system components, force requirements)

Reusability ( R ): possibility of multiple attempts/ targets (loss of functionality, reversibility, deformation)

Risk of additional debris ( Dp ): production of secondary debris (target damage, application of high speeds, style of attachment)

Adaptability ( A ): Attachment surface requirements (surface material/ shape/ structure, geometries)

Breadboard manufacturability ( Bm ): possibility to build a demonstrator using the equipment and devices available at Fraunhofer CML (financial/ time resources, land-based demonstrator)

Since not all of these parameters are considered to have equal effects on the implementation potential of a scenario, weighting factors were established. Using the commonly known practise of a paired comparison, each individual parameter was compared to only one other parameter at a time, reducing the evaluation to the decision whether the first parameter has more (2), equal (1) or less (0) influence compared to the second one. As an example, it is possible to determine that the overall technical feasibility, and therefore, the probability of technology implementation has a higher influence on the success of a scenario than the reusability of the mechanism, since the latter is dependent on the first one. Thus, when the technical feasibility is compared to reusability, it is assigned the number ’2’, indicating a higher influence as depicted in Fig.  3 . In turn, when reusability is considered as the first parameter, it is assigned a ‘0’ when opposite to technical feasibility as it has less influence on the implementation potential. This process was repeated with all parameters, resulting into a comparison matrix that was then used to determine the overall weight of individual parameters. This paired comparison technique is said to produce highly reliable rankings [ 34 ] and was conducted by several aerospace and biomimetics experts with backgrounds in mechanical engineering to determine sensible weighting factors.

Afterwards, the parameters were evaluated with respect to each scenario by assigning them a rank between one and ten (10 = best possible score). Those values were then multiplied by the respective weighting factor for the respective parameter established in the paired comparison and summed up as shown in Eq.  2 , resulting in the final trade-off score (TOS). This score was then used during discussions with ESA representatives to put each scenario’s manufacturability into perspective and decide which three would be investigated further. All of the ten established scenarios are described in the Supplementary Material , where they are presented in descending order according to the analysis results:

Equation 2: Formula to calculate the trade-off analysis score (TOS) for each of the scenarios individually. T: technical feasibility (different parameter than in Sect.  3 , Tc : technical complexity, E : engineering effort, Er : energy requirements, R : reusability, Dp : risk of additional debris production, A : adaptability, Bm : breadboard manufacturability, the character w indicates the respective weighting factor for each of the parameters created by the paired comparison.

Excerpt from the paired comparison matrix, showing the evaluated influence of one parameter on another and vice versa. The sum of all scores (right column) was divided by the sum of all cells in a row, resulting in the respective weighting factor

4.4 Mission parameters and goals

The ten established scenarios presented a very diverse range of ADR options and significantly differed in, e.g., their applicable size of debris as well as their removal strategies. Therefore, mission constraints described in the following were defined and identified for each of the scenarios. The conceptual project boundaries, however, were predefined as aiming to remove large objects in LEO.

Debris type: Describes the type of debris targeted with the described mission. Within the scope of the project, targets were defined according to their mass and dimensions as displayed in Table  1 . Each type was categorized from I to V and lists an exemplary debris object within each category and its current orbital position. As explained above, fragments while listed in the table, were excluded from the conceptual mission design.

Debris condition: While most end-of-life procedures nowadays require the disconnection of batteries and shedding of remaining fuel reserves [ 35 ], these precautions cannot always be executed. Especially when spacecrafts and satellites unexpectedly malfunction, these processes are not carried out. Therefore, if a scenario includes the piercing of the debris surface and penetration further into the object, contained modules as well as their position need to be known to prevent piercing critical or hazardous parts of the debris [ 36 ]. Hence, the conceptualized scenarios presented in the following include an indication if this kind of information is relevant for the successful implementation of the scenario.

Orbit: Some deorbiting strategies such as drag sails requiring atmospheric drag and electromagnetic tethers using Earth’s magnetic field are limited to LEO, where these forces are at work. In GEO, the remaining option is to use propulsion systems to deorbit a target. While this project concentrates on debris orbiting in LEO, possible GEO applications are specified nonetheless.

Number of targets: Depending on their method of capture and deorbiting as well as the encompassed refill/reusability options, some scenarios are not only able to target and remove one object from its current trajectory but continue after achieving its first mission and pursue another target. While targeting multiple objects decreases the cost to benefit ratio significantly, these systems are often complex and are based on slow deorbiting time frames. In addition, while the initial target needs to be prioritized to ensure the success of a conceptualized mission, this parameter indicates whether a created scenario may be applicable to multiple targets instead of a single one.

Number of biomimetic concepts: since the focus of this project are bio-inspired solutions, each scenario also includes an estimate of biomimetic concepts that are combined within one holistic scenario. Low numbers of biomimetic concepts, however, do not indicate insufficient results. It merely gives an indication of the complexity and innovation potential involved.

Type of transport: When defining ADR scenarios, they can be differentiated according to the role of the chaser. In some scenarios, the chaser performs all of the required detection, capturing and deorbiting activities, while in others, it merely presents a transport system able to release submodules responsible for the removal of the target. Here, it is specified for each scenario if the chaser is used as the spacecraft that it is carrying out the mission or if its contained payload takes over this task.

5 Selected biomimetic ADR scenarios

In the following, the three selected scenarios will be presented including all of the encompassed biomimetic concepts and the scenario’s requirements for the mission parameters introduced above (Sect.  4.4 ).

5.1 Scenario A: gecko kit canon

This scenario requires precise data on debris parameters to find an appropriate surface for the subsequent attachment. Therefore, the close-range rendezvous concept has to deliver all necessary information and provide a good fail-to-save approach to enable a successful mission while preventing a collision between chaser and target.

Many invertebrates demonstrate a so-called compound eye, which consists of numerous spherically arranged and cone-shaped sensing units called ommatidia. These units portray high sensitivities, a variety of dimensions, and accept light from narrow angles [ 45 , 46 ]. Hence, the idea of creating artificial compound eyes has been studied before, even in the context of space applications for object detection and space laser communications [ 47 , 48 ]. Taking this concept one step further, the idea is to combine the compound eye and its detection abilities with other biomimetic visual sensing concepts. Sticking to the spherical arrangement of a compound eye, the individual ommatidia are partially replaced by a variety of additional optical sensing options. Those can help overcome limitations of the compound eye detection mechanism itself as well as issues of current detection technologies.

The first technology to be included is based on the lobster’s eyes that provide an especially wide field of view as well as sufficient focusing energies, making it particularly suitable for all-sky- and sun-monitoring. Due to its advantages, it has already been implemented in X-ray telescopes, one of which was employed on a Czech satellite [ 12 , 49 , 50 , 51 ].

Targets orbiting Earth are exposed to an alternating illumination cycle. Hence, the reflected light coming from an object is increasing and decreasing. Therefore, to detect reflected lighting patterns from an object facing away from the sun, the light sensitivity of systems needs to be improved. The white lady spider has developed an efficient neural mechanism that uses temporal and spatial summation of visual stimuli, which allows multiple stimuli to be integrated to capture visual information in dim lighting conditions [ 52 , 53 ]. Using this type of visual and neural mechanism could offer the possibility to increase the detection of poorly reflecting objects in space.

Conventional optical systems for in-orbit detection and identification of target parameters often suffer dynamic illumination conditions or solar glare [ 54 ]. Thus, recent ideas regarding the detection and tracking of space debris have revolved around technologies using different types of light. One of which is using wavelength in the infrared spectrum instead, which several animals are able to detect. Especially certain species of fire beetles have developed a sensing organ equipped with photomechanic sensors able to detect infrared radiation from far distances. The radiation and subsequent temperature increase cause the liquid inside their receptor cells to expand, resulting in a rise in pressure and a subsequent deformation triggering a neural response [ 55 , 56 , 57 , 58 ]. However, Yilmaz and colleagues [ 54 ] argue that a thermal profile of all objects in space has to be established before this technology will find application in the space industry.

Another type of illumination that can be used to avoid issues associated with visible light-based optical sensors is presented by (un)polarized light. In nature, insect pollinators, particularly bees utilize polarized light patterns for navigation, since it is independent from the time of day [ 59 , 60 , 61 ]. Preliminary research has concentrated on determining the polarimetric properties of different commonly used materials for space technologies and establish recognizable patterns to allow a remote characterization and identification of debris [ 20 ].

When it comes to the final approach of a target in space, real-time tracking is important to avoid faulty maneuvers. In nature, the dragonfly is able to pursue its prey within a turbulent environment with distracting stimuli present and still manages to capture a selected target with a 97% success rate. They do so by using so-called small target motion detector neurons, which are very sensitive to target contrast. Hence, they present an efficient and highly adaptable visual processing system that has already been adapted and transferred into tracking algorithms [ 62 , 63 ]. These could improve the processing of collected data from the combined technologies within the compound eye. In addition to the highly efficient target tracking and processing of visual input of the dragonfly, the locust demonstrates another attractive mechanism to avoid collisions between the chaser and the target. Using their lobula giant movement detector neurons, locusts are able to recognize approaching obstacles even in low-contrast conditions or textured backgrounds in motion. Once a collision alert is triggered, the locust can adapt its behavior mid-flight to alter its trajectory and avoid the collision. This collision avoidance mechanism seems very promising and has already been considered for the implementation in smart vehicle technology [ 64 , 65 ].

The biological models mentioned above are depicted in Fig.  4 . Using a compound eye with different optical sensing technologies incorporated that are not only included once but multiple times for the close-range rendezvous in orbit provides an elaborate range of information that can, for example, improve the aiming process at appropriate docking areas on the target. In addition, having multiples of one technology demonstrates a high redundancy and thus, security in case of partial system failure.

Photographs of the biological models included in the compound eye, 1) fly compound eye, 2) lobster 3) white lady spider, 4) locust, 5) fire beetle, 6) bee, 7) dragonfly

Once a sufficient amount of information on the target’s behavior and surface features is determined, the chaser shoots a deorbiting kit towards the target, aiming at a previously determined attachment point. The kit launch system includes a passive energy storage based on the grasshopper hind leg to efficiently propel the kit towards the target. The grasshopper can achieve high catapult forces by slowly contracting one of its muscles while spending only little of its energy. A release triggered by the relaxation of another muscle causes the very fast and strong resulting action [ 66 , 67 ]. Using this kind of system decreases the power consumption by the chaser, but provides an efficient firing mechanism. In addition, the force is sufficient to shoot the kit onto the target, allowing the chaser to maintain a safe distance between itself and the target, thus, decreasing the risk of collision, yet low enough to avoid damages and potentially generate additional debris as well as knocking the target onto a dangerous trajectory. The kit itself is dodecahedronally shaped to provide a high storage volume inside while allowing for large and omnidirectional attachment surface on the outside. The latter is covered with an adhesive mimicking the gecko’s feet to automatically attach whenever it comes into contact with the target. The gecko’s feet make use of a hierarchical compliance of microscopic hairs paired with the van der Waals forces to conform to rough surface and produce sufficient adhesion to enable walking over smooth and vertical surfaces [ 68 ]. Gecko adhesives have been studied extensively and have already been considered partially tested within the space environment [ 69 , 70 , 71 , 72 ]. Thus, they provide a promising concept to gently attach to objects in space. Furthermore, during the shooting process, the kit remains attached to the chaser by a rope that enables the reeling in of the kit in case the first aiming and hitting attempt is not successful. Thus, the approach can be repeated until the mating is achieved. While preliminary tests have resulted in first ideas about velocities and spin rates required in order to form a successful connection [ 69 ], further experiments need to be conducted to determine the adhesive capabilities of gecko adhesive materials when shot towards a target.

Since the chaser in this scenario does not approach the target very closely and will not make a physical connection itself, there is no need to detumble the target or form a pre-attachment before initiating the contact phase. However, in case of dealing with a highly uncontrollable target, the shooting of the kits can be timed so that it adds a counterforce to the spin, thereby slowly reducing its rotational velocity by shooting multiple kits as proposed by Kawamoto and colleagues [ 73 ].

Once the connection to the target is made, the rope from the chaser to the kit is cut, thereby separating the chaser connection to the target. The chaser can go on towards a next target. Inside the deorbiting kit, a drag sail is folded up very small and efficiently based on the folding observed in the leaves of plants [ 74 , 75 ] that can automatically unfold. A thermal release mechanism, passively controlled by external temperature changes (similar to the thermal knife without using current to heat up the system [ 76 ]), triggers the automatic unfolding of the sail. The initial temperature of the target (and all its components) increases through the natural radiation of the sun (depending on orbital location and rotation of the target) that causes the release mechanism to snap and release the sail. All of the remaining biological models for this scenario are depicted in Fig.  5 .

Photographs of the biological models included in scenario A. Picture 1) shows the fly compound eye that incorporates the biomimetic technology concepts presented in Fig.  2 , 2) grasshopper hind leg, 3) gecko feet adhesion, 4) plant leaf folding

The chaser setup designed for this scenario is demonstrated in Fig.  6 . This scenario is designed for the removal of type I targets, since the chaser includes multiple deorbiting kits and can adjust the number of fired kits based on the size of the target. Hence, a sufficiently large drag reduction can be created by multiple drag sails, thereby accelerating the deorbiting despite the rather large mass of the objects. This, however, also limits its application to LEO, since augmenting an object’s atmospheric drag is only possible in this orbital region. Still, it circumvents the additional investment of fuel associated with propulsion deorbiting, since the chaser can remain in its orbit and go on to approach and target new objects.

Sketch Representation of the chaser conceptualized for scenario A. The chaser contains a payload of multiple deorbiting kits (yellow cubes) that can be loaded in front of a catapult that shoots them towards the target once the compound eye (pink) has determined an appropriate position and timing. The kits have a gecko adhesive on their outer surface (green squares) that allows them to stick to the target once they come into contact. If the launch of the kit is successful, the string connecting the target and chaser is cut and the chaser moves away from the target so that the kit can release its drag sail folded up inside

5.2 Scenario B: gecko pad and bee harpoon

Similar to scenario A, this one, too, requires very precise data for the detection and identification of debris parameters, since the chaser approaches the target very closely and performs difficult mating activities. Therefore, the previously introduced compound eye concept (Sect.  5.1 ) is adopted here as well, providing increased safety for the chaser as it offers better determination of appropriate approach lines and maneuvers. In addition to the compound eye, all remaining biological models incorporated in this scenario are depicted together in Fig.  7 .

Furthermore, this scenario encompasses a pre-attachment in form of a kinematically redundant robotic arm, which is a frequently conceptualized idea to detumble a target [ 25 , 26 ]. This robotic arm is modelled after an elephant’s trunk, a highly flexible organ consisting of many muscles, therefore, providing multiple degrees of freedom and high compliance. This demonstrates high maneuverability and adjustability to complex debris motions, shapes and structural features [ 77 , 78 , 79 ]. The length of this robotic arm, however, also determines the maximum possible distance between the chaser and target to initiate the first contact.

To reduce the risk of pushing the target away from the chaser during the capturing phase, a preliminary connection is established using an adhesive pad inspired by the gecko’s feet, similar to the one described above, attached to the artificial trunk as an end-effector. This allows for the establishment of a connection to the target without the application of large forces or velocities. In addition, in case the first connection is not successful or the placement is not ideal, the connection can be undone and the approach repeated until the perfect spot for the preliminary mating is found, since the gecko’s dry adhesion is reversible. Furthermore, Trentlage and Stoll [ 69 ] showed that a foam-like suspension layer underneath the gecko material can enable the capture of curved objects such as the surface shape of rocket upper stages. Hence, underneath the gecko material, a layer of foam inspired by the pomelo fruit’s peel is included. The pomelo demonstrates an open cell foam structure of varying pore size distributed over its thick peel, which protects the fruit inside when falling from trees of up to 10 meters in height. It, therefore, presents excellent impact damping and energy dissipating capabilities. More recently, the beneficial features of the pomelo’s peel have been recognized by the science community and articles have been published studying and modelling the foam-like structure [ 80 , 81 ]. Therefore, it is hypothesized that a bio-inspired material is equally able to dampen the impact between the arm and the target as well as reduce the impact of the preliminary connection onto the target’s trajectory [ 27 ].

Once the preliminary contact is established and the debris cannot escape the hold of the chaser, a harpoon is fired towards the target. On the example of the chameleon’s tongue, the system is charged and fired in a very energy efficient way, reducing the overall energy demand of the mission scenario. Chameleons display the ability to achieve accelerations exceeding 400 \(\hbox {m/s}^2\) due to their rapid elastic recoil of collagen tissue incorporated within its tongue [ 82 ]. While this speed is much higher than the forces necessary to pierce the surface layer of the target and penetrate deep enough into the outer insulation layer to accomplish a firm connection (approximately 20 m/s necessary according to [ 83 ]), it enables the firing of the harpoon from larger distances, which is only limited to the length of the robotic arm. The chameleon achieves such accelerations by slowly contracting one of its tongue muscles that stretches another muscle. The latter is released, while the usual muscle contraction is decoupled, imparting the entire stored energy onto the harpoon [ 82 , 84 ].

The shaft of the harpoon itself presents a conventional harpoon design of a smooth and hollow tube. The tip of the harpoon, however, is modelled after the stinger of the bee that is very sharp and can easily penetrate the surface of the target. In addition, the bee’s stinger demonstrates small hooks at its end to interlock with the skin of the attacked organism [ 85 ]. Thus, this feature is also adapted and transferred to the harpoon’s tip to ensure that it remains in physical contact with the debris’ inner wall. This method creates very high impact forces at the target, which increase the danger of producing additional debris. The required impact force is assumed to be reduced with an additional and preceding contact between chaser and target, since the penetration speed of the harpoon into the target’s wall material can be reduced. While the robotic arm requires a lot of processing, control and navigational power, it is not expected to create a sufficiently rigid attachment to manipulate and manoeuvre very heavy target towards re-entry alone. Hence, the preliminary attachment to the debris with said arm can be used to further prevent the two spacecrafts from drifting apart, while the lengthy processes of aiming and alignment of harpoon and target are conducted. After an appropriate area on the target has been determined, the harpoon is fired. Once a rigid connection is made, the robotic arm can be detached from the target and stored away back in the chaser. The harpoon connected to the chaser with a rope remains inside the debris, while the chaser slowly moves away from the target to create a safe distance between the two. Then, the chaser’s own propulsion system can be used to pull the target behind it and deorbit the object.

The developed chaser for this scenario is depicted in Fig.  8 and shows the combination of all the mentioned techniques in one spacecraft. This scenario allows for the targeting of debris in the LEO and GEO, since it uses its own propulsion system to remove the target from its current trajectory. Moreover, this method presents a short-term deorbiting approach that is very suitable for large targets such as EnviSat in type I. It also does not require a large structure for the capturing but attaches to a comparatively small area on the target itself.

Photographs of the biological models included in scenario B, 1) the compound eye as described above for scenario A, 2) elephant trunk, 3) gecko feet adhesion, 4) pomelo fruit peel dampening, 5) chameleon tongue, and 6) bee stringer.

Sketch Representation of the chaser conceptualized for scenario B. The chaser has a kinematically redundant robotic arm (light blue) with an end-effector consisting of a pomelo fruit peel dampening foam (yellow) and a gecko adhesive surface (dark green). This end-effector displays the component of the chaser that actually attaches to the target. In addition, it has a harpoon firing system (light green) that shoots one harpoon towards the target after the pre-attachment has been successful. The aiming process of the harpoon is done via the compound eye (pink). Using the chaser’s own propulsion system, the target is deorbited

5.3 Scenario C: venus flytrap

Photographs of the biological model included in scenario C, showing a Venus flytrap that has caught a fly.

Contrary to the other two scenarios presented above, the accuracy and precision of data obtained from the target prior to the capturing activities are not as crucial. Hence, one can rely on the existing conventional optical sensing systems such as videometers, advanced video guidance sensors, rendezvous and docking sensors, or laser mappers [ 86 ] to detect and track the target.

The chaser in this scenario has a containment structure attached to one of its sides that is designed after the model of the Venus flytrap that presents two outward bending lobes in one of their bi-stable positions. Carnivorous plants such as the Venus flytrap shown in Fig.  9 exhibit trigger hairs on the inside of their jaw-like lobes. Once prey is attracted through the sweet nectar that is excreted, it settles on the lobes, simultaneously touching several of the flytrap’s hairs. The bending of the hairs triggers an electric signal and initiates a rapid closing of the lobes to capture the prey [ 87 , 88 , 89 ]. Similarly, this principle can be adapted and transferred to debris removal, since it has already been implemented as a small-scale robot [ 88 ], showing the potential of this mechanism for its technical implementation.

The chaser approaches the target against the travel direction, so the hairs make the initial contact once the target has traveled far enough into the lobes. Up to this point, the target is not impacted by the removal mission. When it is in close vicinity to the chaser’s body, however, a sufficient number of the mechano receptor hairs will have been triggered and the bi-stable mechanism autonomously switches from its outward-facing to its inward facing stable position. This allows a complete surrounding of the target without requiring preceding attachment or detumbling actions. Moreover, it does not pose much risk of damaging the target during the capturing process, since it makes very little contact with the debris itself before it is fully contained. Yet, the containment structure is limited in size based on the payload constraints of the carrier rocket. Thus, this scenario is designed to target debris of type III and below. Since this system does not require additional energy for the closing and thus capturing process, it is very energy efficient. In addition, this bi-stable mechanism demands the triggering of multiple stimuli before closing, thus, avoiding inadvertent triggering of the mechanism by dust, particles or small fragments.

Once the target is safely contained by the chaser, it can use its own propulsion system to remove the target from its current orbit. While it is only applicable to one target as the chaser deorbits together with the debris, it allows for the targeting of objects in LEO and GEO, because it uses its own propulsion system and, therefore, does not rely on certain forces or dynamics to be present.

The chaser design for this scenario is depicted in Fig.  10 . Since the chaser’s own propulsion system is used for the deorbiting of the target, the chaser is the active spacecraft performing the removal activities. This scenario is limited in its application to a single target, however, independent from its orbital location. Due to the anticipated constraints for the size of the containment chamber, this scenario will most probably be effective when targeting debris types III and IV. Since this scenario will not make contact with the target at a specific point but will enclose it instead, the conditions of the targets are irrelevant.

Sketch representation of the chaser conceptualized for scenario C. The chaser presents a containment structure that is bi-stable, meaning it has two resting positions and requires the application of energy to change from one into the other position. The green outward facing arcs represent the Venus flytrap’s lobes. The yellow T-shaped structures attached to the lobes are the trigger hairs that are able to receive a stimuli and the blue bars represent the cells that swell due to the triggering of the hairs that causes them to extend and thus force the lobes to switch to their alternate position

6 Conclusions

Within this article, the BIOINSPACED project and its catalogue of valuable biological models for technical implementation in ADR were introduced along with the method of evaluation of collected concepts. The supplementary material contains the complete catalogue of the biomimetic concepts sorted according to their function and ability. It also summarizes the ten initially established scenarios, that hold great value and innovative ideas for biomimetic ADR. The three most promising and innovative conceptual designs for ADR scenarios were presented in this article. All of them display fundamentally different approaches, targeting different debris objects and are applicable in different orbits. All of them, however, display many beneficial traits due to the consideration and integration of biomimetics and many diverse biological models. The BIOINSPACED project has already demonstrated the value that nature’s pool of ideas has to offer when it comes to the development of innovative and improved systems even within the space industry. In the last phase of the project, one of the three presented scenarios will be chosen and implemented as a demonstrator to showcase the functionality of the established concepts and present the potential of biomimetics.

Besides the three selected scenarios within the scope of this project, many of the remaining collected concepts and developed scenarios hold much potential for the application to ADR. Especially the concepts of tactile sensing in space applications to circumvent common limitations of conventional optical systems and passive entanglement of debris (particularly small scale fragments) are of much interest and should be investigated in future. Furthermore, encapsulating the target before making actual contact with it and, therefore, almost completely eliminating the risk of additional debris production present interesting opportunities especially for in-orbit maintenance and servicing.

Ben-Larbi, M.K., et al.: Towards the automated operations of large distributed satellite systems. Part 1: Review and paradigm shifts. Adv. Space Res. 67 (11), 3598–3619 (2021). https://doi.org/10.1016/j.asr.2020.08.009

Article   Google Scholar  

Ben-Larbi, M.K., et al.: Towards the automated operations of large distributed satellite systems. Part 2: Classifications and tools. Adv. Space Res. 67 (11), 3620–3637 (2021). https://doi.org/10.1016/j.asr.2020.08.018

Space Debris Office. ESA’s Annual Space Environment Report. Ed. by ESA Space Debris Office. (2021). https://www.sdo.esoc.esa.int/environment_report/Space_Environment_Report_latest.pdf

Singh, R.A., Yoon, E.S., Jackson, R.L.: Biomimetics: The science of imitating nature. Tribol. Lubr. Technol. 65 (2), 40 (2009)

Google Scholar  

Oeffner, J., Lauder, G.V.: The hydrodynamic function of shark skin and two biomimetic applications. J. Exp. Biol. 215 (Pt 5), 785–795 (2012). https://doi.org/10.1242/jeb.063040

Hwang, J., et al.: Biomimetics: Forecasting the future of science, engineering, and medicine. Int. J. Nanomed. 10 , 5701–5713 (2015). https://doi.org/10.2147/IJN.S83642

Gottlieb Binder GmbH & Co. KG, Produkte Binder: Gecko R Nanoplast R. (24.06.2020). https://www.binder.de/de/produkte/gecko-nanoplast/

Sto SE & Co. KGaA, Fassadenfarbe Lotusan R: Schmutz perlt mit dem Regen ab. (24.06.2020). https://www.sto.de/s/inspiration-information/bionische-produkte/lotusan

Gao, Y., et al.: Deployable wood wasp drill for planetary subsurface sampling. IEEE Aerosp. Conf. (2006). https://doi.org/10.1109/aero.2006.1655756

Menon, C., Ayre, M., Ellery, A.: Biomimetics, a new approach for space systems design. ESA Bull. 125 , 20–26 (2006)

Nakajima, K., Schwarz, O.: How to use the ovipositor drilling mechanism of hymenoptera for developing a surgical instrument in biomimetic design. Int. J. Des. Nat. Ecodyn. 9 (3), 177–189 (2014). https://doi.org/10.2495/DNE-V9-N3-177-189

Daniel, V., et al.: In-orbit commissioning of Czech nanosatellite VZLUSAT-1 for the QB50 mission with a demonstrator of a miniaturised lobster-eye X-Ray telescope and radiation shielding composite materials. Space Sci. Rev. (2019). https://doi.org/10.1007/s11214-019-0589-7

Trentlage, C., et al.: Development and test of an adaptable docking mechanism based on mushroom-shaped adhesive microstructures. In: AIAA (2016). https://doi.org/10.2514/6.2016-5486

de Alba-Padilla, C., Trentlage, C., Stoll, E.: Vision based robot control for grasping space applications using gecko material. In: Proceedings of the Symposium on Advanced Space Technologies in Robotics and Automation, Long Beach, CA, USA, pp. 13–16 (2016)

Ben Larbi, M.K., et al.: Active debris removal for mega constellations: Cubesat possible? https://www.researchgate.net/publication/317722040_Active_Debris_Removal_for_Mega_Constellations_CubeSat_Possible (2017)

Ayre, M.: Biomimetics applied to space exploration. WIT Trans. Ecol. Environ. (2004). https://doi.org/10.2495/DN040591

Egan, P., et al.: The role of mechanics in biological and bio-inspired systems. Nat. Commun. 6 , 7418 (2015). https://doi.org/10.1038/ncomms8418

Banken, E., Schneider, V., Pohl, L., Kniep, J., Ströbel, R., Ben Larbi, M K., Stoll. E., Pambaguian, L., Jahn, C., Oeffner, J.: Assessing Bioinspired Concepts for Space Debris Removal and evaluating their feasibility for simple demonstrator design (2021). In: Proceedings of the 8th European Conference on Space Debris. https://conference.sdo.esoc.esa.int/proceedings/sdc8/paper/24/SDC8-paper24.pdf

Fehse, W.: Automated rendezvous and docking of spacecrafts. Vol 16. Cambridge aerospace series. Cambridge University Press, Cambridge (2003)

Book   Google Scholar  

Pasqual, M.C., Cahoy, K.L.: Active polarimetric measurements for identification and characterization of space debris. IEEE Trans. Aerosp. Electron. Syst. 53 (6), 2706–2717 (2017). https://doi.org/10.1109/TAES.2017.2711718

Maediger, B., et al.: RTES: Robotic technologies for space debris removal. In: International symposium on artificial intelligence, robotics and automation in Space i-SAIRAS, pp. 1–8 (2014)

Peters, T.V., Olmos, D.E.: COBRA contactless detumbling. CEAS Space J. 8 (3), 143–165 (2016). https://doi.org/10.1007/s12567-016-0116-6

Sugai, F., et al.: Detumbling an uncontrolled satellite with contactless force by using an eddy current brake. IEEE/RSJ Int. Conf. Intell. Robots Syst. (2013). https://doi.org/10.1109/IROS.2013.6696440

Voirin, T., Dubois-Matra, O., Kowaltschek, S.: NOMAD: A contactless technique for active large debris removal. In: 63rd International Astronautical Congress (2012)

Liu, Y., et al.: Trajectory planning and coordination control of a space robot for detumbling a flexible tumbling target in post-capture phase. Multibody Syst. Dyn. (2020). https://doi.org/10.1007/s11044-020-09774-6

Wang, M., et al.: Detumbling strategy and coordination control of kinematically redundant space robot after capturing a tumbling target. Nonlinear Dyn. 92 (3), 1023–1043 (2018). https://doi.org/10.1007/s11071-018-4106-4

Nishida, S., Kawamoto, S.: Strategy for capturing of a tumbling space debris. Acta Astronaut. 68 (1–2), 113–120 (2011). https://doi.org/10.1016/j.actaastro.2010.06.045

Bennett, T., Schaub, H.: Touchless electrostatic three-dimensional detumbling of large axi-symmetric debris. J. Astronaut. Sci. 62 (3), 233–53 (2015)

Aglietti, G.S., et al.: RemoveDEBRIS: An in-orbit demonstration of technologies for the removal of space debris. Aeronaut. J. (2020). https://doi.org/10.1017/aer.2019.136

Forshaw, J.L., et al.: The active space debris removal mission remove debris. Part 1: From concept to launch. Acta Astronaut. 168 , 293–309 (2020). https://doi.org/10.1016/j.actaastro.2019.09.002

Aglietti, G.S., et al.: The active space debris removal mission remove debris. Part 2: In orbit operations. Acta Astronaut. 168 , 310–322 (2020). https://doi.org/10.1016/j.actaastro.2019.09.001

Kamps, T., et al.: Systematic biomimetic part design for additive manufacturing. Proced. CIRP 65 , 259–266 (2017). https://doi.org/10.1016/j.procir.2017.04.054

Hoffmann, F., Kesel, A.B.: Biologically inspired optimization of underwater vehicles hull geometries and fin propulsion. In: IEEE (2019)

Lavrakas, Paul J. J.: Encyclopedia of survey research methods. SAGE Publications, Thousand Oaks (2008). https://doi.org/10.4135/9781412963947

Ostrom, C.L., Opiela, J.N.: Orbital debris mitigation and cube- sats. In: 8th European Conference on Space Debris (2021)

Putzar, R., Schäfer, F.: Vulnerability of spacecraft nickel-cadmium batteries to hypervelocity impacts. In: 8th European Conference on Space Debris (2021)

Sommer, S., et al.: Temporal analysis of Envisat’s rotational motion. In: 7th European Conference on Space Debris 2017 ESA SD Vol.7, Nr.1. https://conference.sdo.esoc.esa.int/proceedings/sdc7/paper/437/SDC7-paper437.pdf (2017)

Kucharski, D., et al.: Attitude and spin period of space debris envisat measured by satellite laser ranging. Geoscience and remote sensing. IEEE Trans. 52 , 7651–7657 (2014). https://doi.org/10.1109/TGRS.2014.2316138

Pons, A.P., Noomen, R.: Ariane 5 GTO debris mitigation using natural perturbations. Adv. Space Res. 63 (7), 1992–2002 (2019). https://doi.org/10.1016/j.asr.2018.12.001

arianespace, Ariane 5: User’s Manual. In: Ariane Space Service and Solutions. (2016). https://www.arianespace.com/wp-content/uploads/2011/07/Ariane5_Users-Manual_October2016.pdf

Tadini, P., et al.: Active debris removal of a cosmos-3M second stage b hybrid rocket module. In: VIII International Science and Technology Conference ”Dynamics of Systems, Mechanisms and Machines, pp. 218–237 (2012)

Battie, F., et al.: VEGA launch vehicle upper stage re-entry survivability analysis. IEEE First AESS Eur. Conf. Satell. Telecommun. (ESTEL) (2012). https://doi.org/10.1109/ESTEL.2012.6400200

arianespace.: VEGA: User’s Manual. In: Ariane Space Service and Solutions. (2014). https://www.arianespace.com/wp-content/uploads/2018/05/Vega-Users-Manual_Issue-04_April-2014.pdf

Sylvestre, H., Parama, R.V.R.: Space debris: Reasons, types, impacts and management. Indian J. Radio Space Phys. (IJRSP) 46 (1), 20–26 (2017)

Gonzalez-Bellido, P.T., Wardill, T.J., Juusola, M.: Compound eyes and retinal information processing in miniature dipteran species match their specific ecological demands. Proc. Natl. Acad. Sci. USA 108 (10), 4224–4229 (2011). https://doi.org/10.1073/pnas.1014438108

Deng, Z., et al.: Dragonfly-eye-inspired artificial compound eyes with sophisticated imaging. Adv. Func. Mater. 26 (12), 1995–2001 (2016). https://doi.org/10.1002/adfm.201504941

Zhao, P., et al.: The model research of satellite space laser communication based on compound eye array. Int. Bhurban Conf. Appl. Sci. Technol. (IBCAST) (2017). https://doi.org/10.1109/IBCAST.2017.7868132

Zheng, Y., et al.: Detection of the three-dimensional trajectory of an object based on a curved bionic compound eye. Opt. Lett. 44 (17), 4143–4146 (2019). https://doi.org/10.1364/OL.44.004143

Tamagawa, T., et al.: Multiplexing lobster-eye optics: A concept for wide-field x-ray monitoring. J. Astron. Telesc. Instrum. Syst. 6 (2), 25003 (2020).  https://doi.org/10.1117/1.JATIS.6.2.025003

Hudec, R., Remisova, K.: Biomimetics and astronomical X-ray optics. Contrib. Astron. Obs. Skalnate Pleso 47 , 67–75 (2017).  https://doi.org/10.1117/12.2266591

Hudec, R., et al.: LOBSTER: New space x-ray telescopes. Int. Conf. Space Opt. (2006). https://doi.org/10.1117/12.2308126

Nørgaard, T., Henschel, J.R., Wehner, R.: The night-time temporal window of locomotor activity in the Namib Desert long-distance wandering spider, Leucorchestris arenicola. J. Comp. Physiol. A 192 (4), 365–372 (2006). https://doi.org/10.1007/s00359-005-0072-7

Nørgaard, T., et al.: Vision in the nocturnal wandering spider Leucorchestris arenicola (Araneae: Sparassidae). J. Exp. Biol. 211 (Pt 5), 816–823 (2008). https://doi.org/10.1242/jeb.010546

Yilmaz, Ö., et al.: Thermal analysis of space debris for infrared based active debris removal. Proc. Inst. Mech. Eng. Part G 20 (10), 1–13 (2017). https://doi.org/10.1177/ToBeAssigned

Goris, R.C.: Infrared organs of snakes: An integral part of vision. J. Herpetol. 45 (1), 2–14 (2011). https://doi.org/10.1670/10-238.1

Article   MathSciNet   Google Scholar  

Schneider, E.S., Schmitz, A., Schmitz, H.: Concept of an active amplification mechanism in the infrared organ of pyrophilous melanophila beetles. Front. Physiol. 6 , 391 (2015). https://doi.org/10.3389/fphys.2015.00391

Schmitz, H., et al.: The infrared sensilla in the beetle Melanophila acuminata as model for new infrared sensors. Bioeng. Bioinspired Syst. (2009). https://doi.org/10.1117/12.821434

Martín-Palma, R.J., Kolle, M.: Biomimetic photonic structures for optical sensing. Optics Laser Technol. 109 , 270–277 (2019). https://doi.org/10.1016/j.optlastec.2018.07.079

Foster, J.J., et al.: Bumblebees learn polarization patterns. Curr. Biol. 2412 , 1415–1420 (2014). https://doi.org/10.1016/j.cub.2014.05.007

Lucas, M.A., Chahl, J.S.: Challenges for biomimetic night time sky polarization navigation. Bioinspiration Biomim Bioreplication (2016). https://doi.org/10.1117/12.2219083

Evangelista, C., et al.: Honeybee navigation: Critically examining the role of the polarization compass. Philos. Trans. R. Soc. Lond. B 369 (1636), 20130037 (2014). https://doi.org/10.1098/rstb.2013.0037

Bagheri, Z.M., et al.: Performance of an insect-inspired target tracker in natural conditions. Bioinspiration Biomim. 12 (2), 025006 (2017). https://doi.org/10.1088/1748-3190/aa5b48

Colonnier, F., et al.: A bio-inspired sighted robot chases like a hoverfly. Bioinspiration Biomim. 14 (3), 036002 (2019). https://doi.org/10.1088/1748-3190/aaffa4

Yue, S., et al.: A bio-inspired visual collision detection mechanism for cars: Optimisation of a model of a locust neuron to a novel environment. Neurocomputing 69 (13–15), 1591–1598 (2006). https://doi.org/10.1016/j.neucom.2005.06.017

Keil, M.S., Roca-Moreno, E., Rodriguez-Vazquez, A.: A neural model of the locust visual system for detection of object approaches with real-world scenes. (2018). https://doi.org/10.48550/arXiv.1801.08108

Konez, A., Erden, A., Akkök, M.: Preliminary design analysis of like-grasshopper jumping mechanism. In: The 12th International Conference on Machine Design and Production (2006)

Leng, C.Y.: Biomimicry: Grasshoppers inspired engineering innovation. Int. Robot. Autom. J. 5 , 5 (2017). https://doi.org/10.15406/iratj.2017.02.00020

Kim, S., et al.: Smooth vertical surface climbing with directional adhesion. IEEE Trans. Rob. 24 (1), 65–74 (2008). https://doi.org/10.1109/TRO.2007.909786

Trentlage, C., Stoll, E.: The applicability of Gecko Adhesives in a docking mechanism for active debris removal missions. In: 13th Symposium on Advanced Space Technologies in Robotics and Automation, ASTRA (2015)

Cauligi, A., Chen, T. G., Suresh, S. A., Dille, M., Garcia Ruiz, R., Vargas, A. M., Pavone, M., Cutkosky, M. R.: Design and development of a gecko-adhesive gripper for the astrobee free-flying robot. (2020) http://arxiv.org/abs/2009.09151

Bylard, A., et al.: Robust capture and deorbit of rocket body debris using controllable dry adhesion. IEEE (2017). https://doi.org/10.1109/AERO.2017.7943844

Jiang, H., et al.: A robotic device using gecko-inspired adhesives can grasp and manipulate large objects in microgravity. Sci. Robot. 2 , 1–11 (2017).  https://doi.org/10.1126/scirobotics.aan4545

Kawamoto, S., Matsumoto, K., Wakabayashi, S.: Ground experiment of mechanical impulse method for uncontrollable satellite capturing. In: Proceedings of the 6th International Symposium on Artificial Intelligence and Robotics & Automation in Space (2001). http://robotics.estec.esa.int/i-sairas/isairas2001/papers/paper_as004.pdf

de Focatiis, D.S.A., Guest, S.D.: Deployable membranes designed from folding tree leaves. Philos. Trans. R. Soc. Lond. Ser. A 360 (1791), 227–238 (2002). https://doi.org/10.1098/rsta.2001.0928

Patil, H.S., Vaijapurkar, S.: Study of the geometry and folding pattern of leaves of Mimosa pudica. J. Bionic Eng. 4 (1), 19–23 (2007). https://doi.org/10.1016/S1672-6529(07)60008-0

Konink, T., Kester, G.: Multipurpose holddown and release mechanism (MHRM). In: Proceedings of the 13th European Space Mechanisms and Tribology Symposium (2009)

Zhao, J., et al.: A soft biomimetic module of elephant trunk driven by dielectric elastomers. IEEE (2018). https://doi.org/10.1109/ROBIO.2018.8665147

Behrens, R., et al.: An elephant’s trunk-inspired robotic arm: trajectory determination and control. In: Proceedings of 7th German Conference, pp. 417– 421 (2012)

Yang, Y., Zhang, W.: ET arm: Highly compliant elephant-trunk continuum manipulator. ICIRA (2014). https://doi.org/10.1007/978-3-319-13966-1_29

Bührig-Polaczek, A., et al.: Biomimetic cellular metals-using hierarchical structuring for energy absorption. Bioinspiration Biomim. 11 (4), 045002 (2016). https://doi.org/10.1088/1748-3190/11/4/045002

Ortiz, J., Zhang, G., McAdams, D.A.: A model for the design of a pomelo peel bioinspired foam. J. Mech. Des. (2018). https://doi.org/10.1115/1.4040911

Anderson, C.V., Deban, S.M.: Ballistic tongue projection in chameleons maintains high performance at low temperature. Proc. Natl. Acad. Sci. 107 (12), 5495–5499 (2010). https://doi.org/10.1073/pnas.0910778107

Lappas, V.J., Forshaw, J.L., Visagie, L.: RemoveDebris: An EU low cost demonstration mission to test ADR technologies. (2014). International Astronautical Congress, Sep 2014, Toronto, Canada.

Lu, Z., et al.: A catapult robot with chameleon-inspired multi-body elastic nested system. In: 2017 IEEE International Conference on Robotics and Biomimetics : December 5-8, 2017, Macau SAR, China. (2017)

Meyers, M.A., et al.: The cutting edge: Sharp biological materials. Biol. Mater. Sci. 60 , 19–24 (2008). https://doi.org/10.1007/s11837-008-0027-x

Oumer, N.W., Panin, G.: Camera-based tracking for rendezvous and proximity operation of a satellite. In: Bordeneuve-Guibé, J., Drouin, A., Roos, C. (eds.) Advances in Aerospace Guidance, Navigation and Control, pp. 625–638. Springer International Publishing, Cham (2015)

Kreuzwieser, J., et al.: The Venus flytrap attracts insects by the release of volatile organic compounds. J. Exp. Bot. 65 (2), 755–766 (2014). https://doi.org/10.1093/jxb/ert455

Shahinpoor, M.: Biomimetic robotic Venus flytrap (Dionaea muscipula Ellis) made with ionic polymer metal composites. Bioinspiration Biomim. 6 (4), 046004 (2011). https://doi.org/10.1088/1748-3182/6/4/046004

Forterre, Y., et al.: Mechanics of venus ”Flytrap Closure”. In: XXI ICTAM (2004)

Download references

Acknowledgements

The BIOINSPACED study has received funding from the European Space Agency under grant agreement no. 4000130585 “Biomimicry (Biomimetics) for space debris mitigation” in the frame of ESA’s Discovery & Preparation studies. Special thanks goes to all brainstorming workshop participants, who contributed to the collection of biomimetic concepts and ideas and the remaining project members, especially our student assistance Lotte Pohl and Jula Kniep for outstanding support and organisation.

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Fraunhofer Centre for Maritime Logistics and Services CML, Am Schwarzenberg-Campus 4, Building D, 21073, Hamburg, Germany

E. Banken, V. E. Schneider & J. Oeffner

Technische Universität Braunschweig, Institute of Space Systems, Hermann-Blenk-Str. 23, 38108, Braunschweig, Germany

M. K. Ben-Larbi

ESA-ESTEC, Keplerlaan 1, PO Box 299, 2200 AG, Noordwijk, The Netherlands

L. Pambaguian

Department of Aeronautics and Astronautics, Chair of Space Technology, Technical University Berlin, Marchstr. 12-14, 10587, Berlin, Germany

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to J. Oeffner .

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 302 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Banken, E., Schneider, V.E., Ben-Larbi, M.K. et al. Biomimetic space debris removal: conceptual design of bio-inspired active debris removal scenarios. CEAS Space J 15 , 237–252 (2023). https://doi.org/10.1007/s12567-022-00438-z

Download citation

Received : 05 July 2021

Revised : 31 January 2022

Accepted : 28 February 2022

Published : 22 April 2022

Issue Date : January 2023

DOI : https://doi.org/10.1007/s12567-022-00438-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Biomimetics
• Bioinspired space solutions
• Active space debris removal
• Space debris remediation
• Find a journal
• Publish with us
• Track your research

Game Changer: Triggers and Effects of an Active Debris Removal Market

Download Paper

The recent increase in space actors has revived discussions about active debris removal (ADR) systems, which are needed to sustain the space environment. Space debris has been a growing concern for over four decades, particularly as more entities gain access to space. The oldest piece in orbit, the U.S. Vanguard 1 satellite, was launched in 1958, and will remain in MEO for at least the next 200 years. ADR systems and development testing, while welcomed, is currently hamstrung by a lack of market demand due to technical, operational, and political challenges. However, a triggering event could increase market demand for ADR services, transforming them from a niche market to a staple in the space industry.

Author: Christopher May

Learn more about this topic from experts in episode 55 of #TheSpacePolicyShow " Active Debris Removal" .

Download this paper at: https://csps.aerospace.org/papers/game-changer-triggers-and-effects-active-debris-removal-market

NTRS - NASA Technical Reports Server

Available downloads, related records.