3.12. Annex C: modified questions in the process questionnaire

The questionnaire is performed by a list of questions applicable for each phase of the development and manufacturing in order to estimate the process factor \(\Pi_{\text{process}}\) for quality and technical control over reliability in the product life cycle as defined in Section Section 3.4.1. However, some of the questions are not perfectly applicable to space applications. This is why this present annex contains the modified questions of the questionnaire adapted to space applications according to the content of Section Section 3.4.3.2.4. Recommendations for the satisfaction level of the modified and non-modified questions of the questionnaire are proposed in Section 3.11.

Questions n°14, n°71 and n°114 are modified only for the “Operation and maintenance” phase and are not modified for the other phases. Other questions are modified for all phases of use.

The question n°14 is modified for the “Operation and maintenance” phase according to the following table. It is not modified for “Integration into equipment” and “Integration into system” phases.

Table 3.12.1 Modified question n°14 for Operation and Maintenance.

Recommendation	Perform final inspections and tests on launch pad.
N°	14
Phases in which the recommendation is applicable	OPERATION AND MAINTENANCE
Weight	10.4
Further description	Perform all final inspections and tests on launch pad in accordance with the quality plan and/or written procedures. The quality plan and/or procedures for final inspections and tests on launch pad must require that all specified inspections and tests, including those defined for product reception, are carried out and that the results are conforming with requirements. Before shipment, make sure that: All activities specified in the quality plan and/or written procedures have been satisfactory accomplished, The associated data and documentation are available and accepted.
Audit question	Have all final inspections and tests on launch pad been carried out in accordance with the quality plan and/or written procedures?
Level 1	No final inspection or test on launch pad.
Level 2	Final inspections and tests on launch pad are carried out, but they are not described in strict procedures or in a quality plan.
Level 3	Final inspections and tests on launch pad are carried out, They are described in strict procedures or in a quality plan.
Level 4	Final inspections and tests on launch pad are carried out in accordance with the quality plan and/or written procedures. The quality plan and/or procedures for final inspections and tests on launch pad require that all specified inspections and tests, including those specified for product reception or during its manufacturing, are done and that the results are conforming with the requirements. It is checked before shipment that: All activities specified in the quality plan and/or in written procedures have been satisfactorily accomplished Data and the associated documentation are available (follower sheet type document that records the configuration, operations carried out and observed anomalies) and accepted.

The question n°28 is modified for the “Specification” phase according to the following table.

Table 3.12.2 Modified question n°28.

Recommendation	Define product failure.
N°	28
Phases in which the recommendation is applicable	SPECIFICATION
Weight	10.3
Further description	Precisely define what will be considered as a product failure (possibilities of acceptable degraded modes).
Audit question	What is considered as a product failure?
Level 1	No description of the product failure was defined in the contract. The customer has not given any list of feared events. The customer has not defined any degraded mode. The manufacturer has not defined these elements for his study.
Level 2	The manufacturer has produced the description of the product failure and (or) the list of feared events, and (or) product degraded modes without formal validation by the customer.
Level 3	The manufacturer has produced the description of the product failure and (or) the list of feared events, and (or) product degraded modes with formal validation by the customer.
Level 4	Product failures are perfectly identified in the contract. The list of feared events is supplied in the contract. Degraded modes are also described in the contract.

The question n°39 is modified for the “Design” phase according to the following table.

Table 3.12.3 Modified question n°39.

Recommendation	Have and manage a table of skills required by activity, including names.
N°	39
Phases in which the recommendation is applicable	DESIGN
Weight	24.5
Further description	Make sure that the skills required for an activity are assigned to the operators and periodically verify that training is appropriate for the activities.
Audit question	Is there any management of skills?
Level 1	No monitoring of suitability for training.
Level 2	The operators have skills required for an activity but are not monitored in training
Level 3	The operators have training required for an activity, but there is no periodic verification that training is suitable for the activities.
Level 4	Training is periodically monitored and updated. There are regular evaluations of the suitability of training to satisfy the company’s objectives.

The question n°44 is modified for the “Design” phase according to the following table.

Table 3.12.4 Modified question n°44.

Recommendation	Write a collection of business recommendations dealing with manipulation and storage operations at the user.
N°	44
Phases in which the recommendation is applicable	DESIGN
Weight	7.7
Further description	Make sure that there is a list of business recommendations on manipulation and storage operations at the user. This collection must be enriched by feedback from operations.
Audit question	Is there a collection of business recommendations for manipulation and storage operations at the customer?
Level 1	No collection of recommendations and no procedures for processing of feedback from operations.
Level 2	There is non formalised and unmanaged collection of recommendations. Feedback from operations is dealt but not systematically.
Level 3	There is a formalised collection of recommendations but it is not necessarily applicable to the project (not referenced to the project) and is not validated. Feedback from operations formalised in a base that is not managed and is not much used in design.
Level 4	Formalised collection of recommendations, validated and referenced to the project. Formalised and validated feedback from operations, referenced to the project, useable and acting as design input data to improve reliability.

The question n°61 is modified for the “Design”, “Manufacturing of board or subassembly” and “Integration into equipment” phases according to the following table.

Table 3.12.5 Modified question n°61.

Recommendation	Identify and implement means of protecting subassemblies.
N°	61
Phases in which the recommendation is applicable	DESIGN, MANUFACTURING OF BOARD OR SUBASSEMBLY, INTEGRATION INTO EQUIPMENT
Weight	7.3, 7.3, 7.3
Further description	List and implement protection means to avoid reducing the reliability of the subassembly.
Audit question	Have means of protecting subassemblies during some production activities been identified and implemented?
Level 1	No particular protection means is identified.
Level 2	Protection means are identified but are partially applied in different activities.
Level 3	Protection means are identified and their application is verified.
Level 4	Protection means are identified following management of lessons-learnt and experience and their application is verified.

The question n°68 is modified for the “Support process activities” phase according to the following table.

Table 3.12.6 Modified question n°68.

Recommendation	Identify risks related to Reliability at subcontractors.
N°	68
Phases in which the recommendation is applicable	SUPPORT PROCESS ACTIVITIES
Weight	7.2
Further description	During the design phase (with the subcontractor) identify risks associated with the reliability of the subcontracted product.
Audit question	Have risks related to reliability of products at subcontractors been identified?
Level 1	No reliability risk analysis is made with the subcontractor during the design phase (no specific provision)
Level 2	No reliability risk analysis is made with the subcontractor during the design phase, but a risk identification was made during the job. These risks are not managed.
Level 3	The reliability risk analysis was made during the design phase and is explained in a non-formal analysis. These risks are not managed.
Level 4	The reliability risk analysis was made during the design phase and is described in a formal document. Identified risks are regularly updated.

The question n°71 is modified only for the “Operation and maintenance” phase according to the following table. This question is not modified for the “Integration into equipment” and “Integration into system” phases.

Table 3.12.7 Modified question n°71 for Operation and Maintenance.

Recommendation	Control documentation.
N°	71
Phases in which the recommendation is applicable	OPERATION AND MAINTENANCE
Weight	5.6
Further description	Store and keep product and process documentation available during operation process. Keep the list of documentation updated with respect to evolutions (ex: number of reconfigurations, number of station keeping manoeuvers, …) Train operators to follow instructions and recommendations. The whole documentation related to operations has to be up to date and available for operators. When documents are provided, analyse the validity of this product documentation. Be in possession of process control documentation. Specify technical documentation for each process (ex: avoidance manoeuvers, reconfiguration from ground, …). Provide this technical documentation and make it usable.
Audit question	Is documentation well controlled? Does it take account of all operational configuration changes?
Level 1	No documentation specific to the operational configuration or processes, there are no plans to provide special documentation.
Level 2	Documentation specific to the operational configuration or processes does exist, however it is not always effectively updated, there is no analysis about the validity of documents.
Level 3	Documentation specific to the operational configuration or processes does exist, it is updated periodically in a planned manner, the validity of the documents used is not analysed.
Level 4	Documentation specific to the operational configuration or processes does exist, it is updated periodically in a planned manner, the validity of the documents used is analysed. Precise procedures are applied for storage and preservation of the documentation.

The question n°72 is modified for the “Operation and maintenance” phase according to the following table.

Table 3.12.8 Modified question n°14 for Operation and Maintenance.

Recommendation	Control product monitoring and maintainability in operation.
N°	72
Phases in which the recommendation is applicable	OPERATION AND MAINTENANCE
Weight	17.6
Further description	Control the capability of products to detect failures, control failure detection means, allow maintainability in operation.
Audit question	How are product monitoring and maintainability in operation controlled?
Level 1	No monitoring or maintainability performed
Level 2	There is a minimum surveillance by alarms
Level 3	The FDIR process is mainly automatic.
Level 4	The FDIR process requires both automatic reaction and manual intervention from ground. The FDIR principles are described in the FDIR User Manual. The operators have been trained for such procedures.

The question n°76 is modified for the “Support process activities” phase according to the following table.

Table 3.12.9 Modified question n°76.

Recommendation	Control monitoring and measurement devices, metrology of measurement instruments and industrial means.
N°	76
Phases in which the recommendation is applicable	SUPPORT PROCESS ACTIVITIES
Weight	7.8
Further description	Control monitoring and measurement devices, the metrology of measurement instruments and industrial means. Control the verification, calibration and rating of measurement instruments and test and trial benches used by the company. APMs are tied to national standards.
Audit question	What procedure is there to control monitoring and measurement devices, the metrology of measurement instruments and industrial means?
Level 1	There is no verification, calibration and rating procedure for measurement instruments and test and trial benches in the company.
Level 2	There is a verification, calibration and rating procedure for measurement instruments and test and trial benches in the company, but without any evidence of application.
Level 3	There is a verification, calibration and rating procedure for measurement instruments and test and trial benches in the company, and it is applied.
Level 4	Verification, calibration and rating of measurement instruments and test and trial benches used by the company is controlled (accreditation, certification, etc.); APMs are tied to national standards.

The question n°104 is modified for the “Support process activities” phase according to the following table.

Table 3.12.10 Modified question n°104.

Recommendation	Appoint a person responsible for reliability studies.
N°	104
Phases in which the recommendation is applicable	SUPPORT PROCESS ACTIVITIES
Weight	8.5
Further description	Appoint a person responsible for reliability for each project, who will guarantee that product reliability objectives are achieved. This person shall report on progress with studies and problems encountered.
Audit question	Has a person responsible for reliability studies been appointed?
Level 1	No person responsible for reliability studies has been identified.
Level 2	There is a person responsible for reliability studies in practice, but no record of his appointment is available or does not have been trained or does not have experience in the reliability field.
Level 3	A reliability study manager has been formally named, he has been trained and has experience in the reliability field.
Level 4	A reliability study manager has been named and integrated into the project, he is trained and has the required experience in the field. He makes regular reports on progress with studies in meetings or in reports.

The question n°114 is modified only for the “Operation and maintenance” phase according to the following table. This question is not modified for the “Integration into equipment” and “Integration into system” phases.

Table 3.12.11 Modified question n°114 for Operation and Maintenance.

Recommendation	Possess a check up procedure.
N°	114
Phases in which the recommendation is applicable	OPERATION AND MAINTENANCE
Weight	5.7
Further description	The state of the satellite is regularly checked through tests implemented in the operation procedure.
Audit question	Is the state of the satellite (and its subsystems) checked?
Level 1	No check is scheduled and performed.
Level 2	No check is scheduled but can be performed if deemed necessary.
Level 3	Checks are scheduled and performed following a procedure. There is no recordings of what is done.
Level 4	Checks are scheduled, performed following a procedure. Every intervention is documented. The operator is trained for this procedure.

The question n°157 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.12 Modified question n°157 for Ruggedising.

Recommendation	Provide training and manage maintenance of skills for implementation of the product
N°	157
Phases in which the recommendation is applicable	RUGGEDISING
Weight	7
Further description	Train users to make sure that the product is always used correctly
Audit question	Have users (for use) received training on the product? Is this training repeated and updated to satisfy needs?
Level 1	No training associated with the product
Level 2	There is some initial training or training of only some users
Level 3	There is a complete training but with no skills management
Level 4	There is a complete training. Skills management assures that all users have been trained and that the training is up to date

The question n°158 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.13 Modified question n°158 for Ruggedising.

Recommendation	Check that procedures specific to the product and rules specific to businesses are respected by an appropriate monitoring system
N°	158
Phases in which the recommendation is applicable	RUGGEDISING
Weight	7
Further description	Implement means to supervise and control users in use of the product, to be able to identify deviations and deal with them.
Audit question	Are inspection means (process, recording means) sufficient for the supplier to assure that rules for the use of the product are well respected by users?
Level 1	No inspection means
Level 2	Existence of a few monitoring and inspection means
Level 3	Existence of non-exhaustive or informally used inspection means. Deviations are not systematically dealt with.
Level 4	There are complete and formalised inspection means. Deviations are dealt with.

The question n°159 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.14 Modified question n°159 for Ruggedising.

Recommendation	Design dependable electrical protection devices.
N°	159
Phases in which the recommendation is applicable	RUGGEDISING
Weight	4
Further description	Identify protection needs. Design electrical protection devices. Make sure that they are testable.
Audit question	How are electrical protection devices designed?
Level 1	Operating dependability principles are not applied to electrical protection devices
Level 2	Operating dependability principles are applied to electrical protection devices in some cases
Level 3	Operating dependability principles are applied to electrical protection devices. There are verifications that these devices are operating correctly throughout the life of the product.
Level 4	Operating dependability principles are applied to electrical protection devices. There are verifications that these devices are operating correctly throughout the life of the product. This approach is described in a procedure.

The question n°160 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.15 Modified question n°160 for Ruggedising.

Recommendation	Study and handle risks of the product under test being deteriorated by failures of its test means
N°	160
Phases in which the recommendation is applicable	RUGGEDISING
Weight	4
Further description	Minimise the risk of deterioration by taking it into account in the design of the test means and the tested unit, develop appropriate prevention means.
Audit question	How are risks of the product under test being deteriorated by failure of its test means dealt with?
Level 1	No study of failures of the test means
Level 2	Some known failures are taken into account
Level 3	These risks are analysed in the design of the test means and unit tested. Appropriate prevention means are set up.
Level 4	These risks are analysed in the design of the test means and unit tested. Appropriate prevention means are set up. This approach is described in a procedure.

The question n°161 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.16 Modified question n°161 for Ruggedising.

Recommendation	Identify and use appropriate prevention means of preventing reasonably predictable aggressions
N°	161
Phases in which the recommendation is applicable	RUGGEDISING
Weight	4
Further description	Search and prevent effects of reasonably predictable aggressions (UV, light, radiations, single atomic oxygen, etc.)
Audit question	Are reasonably predictable aggressions taken into account?
Level 1	Aggressions are not taken into account
Level 2	Well known aggressions are taken into account
Level 3	Aggression cases are searched for and are taken into account
Level 4	Aggression cases are searched for and are taken into account. This approach is described in a procedure or a standard

The question n°163 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.17 Modified question n°163 for Ruggedising.

Recommendation	Include production and storage environments in the product environment specifications
N°	163
Phases in which the recommendation is applicable	RUGGEDISING
Weight	4
Further description	Extend environment specifications so that they also cover production and storage situations (and not only usage cases). The controlling parameter for storage could be the duration.
Audit question	How are production and storage environments taken into account in the product environment specification?
Level 1	Production and storage environments are not specified
Level 2	Production and storage environments are taken into account if they are known
Level 3	Production and storage environments are taken into account in environment specifications
Level 4	Production and maintenance environments are systematically taken into account. Environments are formalised in a documentation

The question n°167 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.18 Modified question n°167 for Ruggedising.

Recommendation	Carry out a process analysis of implementation operations
N°	167
Phases in which the recommendation is applicable	RUGGEDISING
Weight	4
Further description	Carry out a process FMECA (Failure Modes, Effects and Criticality Analysis) for implementation operations
Audit question	How are risks of errors in carrying out implementation operations analysed?
Level 1	No analysis of risks of errors in carrying out operations
Level 2	Well known anomalies and drifts are collected and used in the design of the product
Level 3	The process FMECA for implementation operations is done in some cases and is used for the product design
Level 4	The process FMECA for implementation operations is systematically done and is used for the product design

The question n°169 is modified for the “Ruggedising” phase according to the following table.

Table 3.12.19 Modified question n°169 for Ruggedising.

Recommendation	Write complete procedures for all product implementation operations
N°	169
Phases in which the recommendation is applicable	RUGGEDISING
Weight	7
Further description	Provide users with documentation describing procedures to be applied for each situation
Audit question	Is there any documentation that describes all product implementation operations?
Level 1	No documentation, or superficial documentation
Level 2	There is documentation covering some of the needs
Level 3	There is complete documentation, but its exhaustiveness cannot be proven
Level 4	The documentation exists and is complete. The composition of the documentation is described in a procedure that guarantees that it is complete