从NASA的人那里看system engineering，感觉像是在讨论数学建模的问题，于是找了本数学建模的本质——CAE的方法论也在数学建模那里！

《The Nature of Mathematical Modeling》

解析解，数值解，有限单元，偏微分方程等等。这都是CAE的概念！

This is a book about the nature of mathematical modeling and about the kinds of techniques that are useful for modeling (both natural and otherwise). It is oriented towards simple, efficient implementations on computers. 大家的目的算是一致了！

The text has three parts. 解析解的问题The first covers exact and approximate analytical techniques (ordinary differential and difference equations, partial differential equations, variational principles, stochastic processes); 数值分析的问题the second, numerical methods (finite differences for ODEs and PDEs, finite elements, cellular automata); and the third, 实验解的问题model inference based on observations (function fitting, data transforms, network architectures, search techniques, density estimation, filtering and state estimation, linear and nonlinear time series).

这个分类本质上跟altair的那本the practice of

Each of these essential topics would be the worthy subject of a dedicated text, but such a narrow treatment obscures the connections among old and new approaches to modeling. By covering so much material so compactly, this book helps bring it to a much broader audience. Each chapter presents a concise summary of the core results in an area, providing an accessible introduction to what they can (and cannot) do, enough background to use them to solve typical problems, and then pointers into the specialized research literature. The text is complemented by a Website and extensive worked problems that introduce extensions and applications. This essential book will be of great value to anyone seeking to develop quantitative and qualitative descriptions of complex phenomena, from physics to finance.

Professor Neil Gershenfeld leads the Physics and Media Group at the MIT Media Lab and codirects the Things That Think research consortium. His laboratory investigates the relationship between the content of information and its physical representation, from building molecular quantum computers to building musical instruments for collaborations ranging from Yo-Yo Ma to Penn & Teller. He has a BA in Physics from Swarthmore College, was a technician at Bell Labs, received a PhD in Applied Physics from Cornell University, and he was a Junior Fellow of the Harvard Society of Fellows.

n5321 | 2025年7月14日 23:18

近期最佳paper！一个土耳其人，有博世工作经验，做学术，然后拿了米帝佐治亚理工学院的博士,中间做了NASA的project。兼工程&学术背景的人写的东西还是不一样一点。

它有几个基本点：工程师难以全面准确把握产品内部的物理关系——这是他想要解决的问题——工程师的认知问题！

仿真分析是航天业的常规操作，他有经验，所以可能用了一整套航天业的语言来描述这个问题。整体的思路是完全同意，本质上也没有超出我的认知！相当多的一部分人天天还盯着单元类型，自由度等等问题，算是没有进入实战阶段！不过他本质上还是在用概率论来实现目标。

简单一点说：仿真结果是很容易出错，或者说它就是错的，但是它是有用的。那错的原因在哪里，这个paper的分类是不错的，一个是工艺问题，一个是认知的问题，理论上来看，你不可能建立一个完全物理镜像的，跟实物一致的仿真模型，一个是工艺的原因，他有很多偶然性，有公差在！第二个是你对真实世界的还原是存在偏差的，科学的关系都是对现实世界抽象以后获得的数学表达式，你现在要把所有的表达式东西都还原，应用到工程实际中去其实是不可能的。然后有用的原因他讲的也不错。就是仿真的结果跟最后实际使用的结果一致。怎么实现？中间就需要对模型进行很多的处理！怎么处理，去尝试！——算是Swanson那个台词的一个大的补充，跟自己的思路基本上也一致。简单说就是在电脑上试错，把设计参数跟性能结果之间的应变关系基本搞清楚！然后再做模型的处理，最后让计算机实验跟物理实验的结果一致！——达到预测价值！

但是传统行业的人对IT理解不够，如果他懂数据库，知道SQL，再添加上一点data visulization，应该要厉害更多！

Complex engineering systems require the integration of sub-system simulations and the calculation of system-level metrics to support informed design decisions. This paper presents a methodology for designing computational or physical experiments aimed at mitigating system-level uncertainties.   工程问题的解决方案这样子计算两个分类。分析解，数值解，实验解算是做三个分类。

The approach is grounded in a predefined problem ontology, where physical, functional, and modeling architectures（这三个词已经屏蔽了很多人！） are systematically established.——仿真分析的框架已经定下来了。目的，领域，模型都已经有了！By performing sensitivity analysis using system-level tools, critical epistemic uncertainties can be identified. 这个地方是我所谓的工艺参数！Based on these insights, a framework is proposed for designing targeted computational and physical experiments to generate new knowledge about key parameters and reduce uncertainty.这个framework是有一点兴趣的！跟我的会有什么不一样吗？

 The methodology is demonstrated through a case study involving the early-stage design of a Blended-Wing-Body (BWB) aircraft concept, illustrating how aerostructures analyses can support uncertainty mitigation through computer simulations or by guiding physical testing. The proposed methodology is flexible and applicable to a wide range of design challenges, enabling more risk-informed and knowledge-driven design processes.

1Introduction and Background

The design of a flight vehicle is a lengthy, expensive process spanning many years. With the advance in computational capabilities, designers have been relying on computer models to make predictions about the real-life performance of an aircraft. 飞行器的设计是一个漫长而昂贵的过程，需要耗费数年时间。随着计算能力的进步，设计人员一直依赖计算机模型来预测飞机的实际性能。这个行业是唯一完全CAE覆盖的行业，而且覆盖的特别早！However, the results obtained from computational tools are never exact due to a lack of understanding of physical phenomena, inadequate modeling and abstractions in product details [1, 2, 3]. 每一个人都关心的准确性问题！跟求解器不同，CAE厂家的台词不同，这是工程师视角：物理没学好，模型不准确导致仿真结果错误！ The vagueness in quantities of interest is called uncertainty. The uncertainty in simulations may lead to erroneous predictions regarding the product; creating risk.仿真没做好，仿真中的不确定会导致风险！——工程师的核心关注点！

Because most of the cost is committed early in the design [4], any decision made on quantities involving significant uncertainty may result in budget overruns, schedule delays and performance shortcomings, as well as safety concerns. 盲人摸象的设计带过来的几个风险词说得也不错！超预算，延期，性能差，安全隐患。

Reducing the uncertainty in simulations earlier in the design process will reduce the risk in the final product. The goal of this paper is to present a systematic methodology to identify and mitigate the sources uncertainty in complex, multi-disciplinary problems such as aircraft design, with a focus on uncertainties due to a lack of knowledge (i.e., epistemic).

我取的名字是如何获得上帝视角——omniscience！

1.1The Role of Simulations in Design

Computational tools are almost exclusively used to make predictions about the response of a system under a set of inputs and boundary conditions [5].

工程师确实是应该把CAE厂家在80年代奠定的一系列台词丢掉！这个台词就带一点the first principle的味道！CAE是预测工程方案与设计意图是否匹配！
At the core of computational tools lies a model, representing the reality of interest, commonly in the form of mathematical equations that are obtained from theory or previously measured data. 数学建模  How a computer simulation represents a reality of interest is summarized in Figure 1. Development of the mathematical model implies that there exist some information about the reality of interest (i.e., a physics phenomenon) at different conditions so that the form of the mathematical equation and the parameters that have an impact on the results of the equation can be derived. The parameters include the coefficients and mathematical operations in the equations, as well as anything related to representing the physical artifact, boundary and initial conditions, system excitation [6].  里面涉及到大量的参数！一个参数错了，结果就不对！

A complete set of equations and parameters are used to calculate the system response quantities (SRQ). Depending on the nature of the problem, the calculation can be straightforward or may require the use of some kind of discretization scheme.

If the physics phenomenon is understood well enough such that the form of the mathematical representation is trusted, a new set parameters in the equations (e.g., coefficients) may be sought in order to better match the results with an experimental observation. This process is called calibration. 本质上我自己挑的validation不够准确，还是用这个词的描述更好！With the availability of data on similar artifacts, in similar experimental conditions; calibration enables the utilization of existing simulations to make more accurate predictions with respect to the measured "truth“ model.  这个就毫无新意了!

Most models are abstractions of the reality of interest as they do not consider the problem at hand in its entirety but only in general aspects that yield useful conclusions without spending time on details that do not significantly impact the information gain. 这是every mocel is wrong ,but some are useful的另外一个讲法！ Generally, models that abstract fewer details of the problem are able to provide more detailed information with a better accuracy, but they require detailed information about the system of interest and external conditions to work with. Such models are called high-fidelity models. Conversely, low-fidelity models abstract larger chunks of the problem and have a quick turnaround time. They may be able to provide valuable information without so much effort going into setting up the model; unlike high-fidelity models, they can generally do it with very low computational cost. The choice of the fidelity of the model is typically left to the practitioner and depends on the application.

搭配看Swanson的一段台词看：

Let me go slightly sideways on that. When I first started teaching ANSYS, I said always start small. If you're going to go an auto crash analysis, you have one mass rep a car and one spring. Understand that? Then you can start modifying. You can put the mass on wheels and allow it to rotate a little bit when it hits a wall, and so on. So each, each new simulation should be a endorsement of what's happened before. Not a surprise. Yeah, if you get a surprise and simulation, you didn't understand the problem. Now, I'm not quite I think that relates to what your question was. And that is basically start at the start small and work your way up. Don't try to solve the big problem without understanding all the pieces to it.

只是同一个意思的不同表达

A few decades ago, engineers had to rely on physical experiments to come up with new designs or tune them as their computational capabilities were insufficient. 做样优化 Such experiments that include artifacts and instrumentation systems are generally time consuming and expensive to develop. In the context of air vehicle design, design is an inherently iterative process and these experiments would need to be rebuilt and reevaluated. Therefore, while they are effective for evaluation purposes, they cannot be treated as parametric design models unless they have been created with a level of adjustability. With the advance of more powerful computers and widespread use of them in all industries（ANSYS都卖身了，明显行业没渗透出去）, engineers turned to simulations to generate information about the product they are working on. Although detailed simulations can be prohibitively expensive in terms of work-hours and computation time; the use of computer simulations is typically cheaper and faster than following a build-and-break approach for most large-scale systems.

As the predictions obtained from simulations played a larger part in the design, concepts such as “simulation driven design" has been more prominent in many disciplines. [7] If the physics models are accurate, constructing solution environments with very fine grids to capture complex physics phenomena accurately become possible.准确分网是CAE分析结果准确的一个因数而已，只是影响收敛性！建立准确的模型！基本上来说准确的模型，准确的仿真结果！ The cost of making a change in the design increases exponentially from initiation to entry into service [8]. If modeling and simulation environments that accurately capture the physics are used in the design loop, it will be possible to identify necessary changes earlier.正确的废话 Because making design changes later may require additional changes in other connected sub-systems, it will lead to an increase in the overall cost [9].

1.2Modeling Physical Phenomena

When the task of designing complex products involves the design of certain systems that are unlike their counterparts or predecessors, the capability of known physics-based modeling techniques may come short.经验的价值，一个全新的产品不太能建立准确的分析模型！ For example, the goal of making predictions about a novel aircraft configuration aircraft, a gap is to be expected between the simulation predictions and the measurements from the finalized design——仿真结果跟测试结果基本上会存在差异——定量的差异！. If the tools are developed for traditional aircraft concepts (e.g., tube and wing configurations) there might even be a physics phenomenon occurring that will not be expected or captured——甚至有定性的差异！预测不准的源头！. Even if there is none, the accuracy of models in such cases are still to be questioned.——就算仿真跟实际检验结果一致，也可能模型不准！仿真经验丰富的人知道这不是假话！ There are inherent abstractions pertaining to the design and the best way to quantify the impact of variations in the quantities of interest by changing the geometric or material properties is by making a comparative assessment with respect to the historical data 性能设计上面需要PDM. However, in this case, historical data simply do not exist. 没有已知的数据可以做参考

Because of a lack of knowledge or inherent randomness, the parameters used in modeling equations, boundary/initial conditions, and the geometry are inexact, i.e., uncertain. 本质上不确定性是无法消除的，原因他认识是两类，工艺加无知！没有实验室的行业无法做CAE的calibration。 The uncertainty in these parameters and the model itself（模型参数本质上是不确定的！）, manifest themselves as uncertainty in the model output. As mentioned before, any decision made on uncertain predictions will create risk in the design. John A. Swanson（ansys founder）所谓的分析的时候没有意外！他们本质上把CAE当做一个定量的工具，trench是了然的，但是到哪个点不清楚，需要准确判断，才需要CAE。In order to tackle the overall uncertainty, the sources of individual uncertainties must be meticulously tracked （some are useful的本质原因，风险是已知的，类似身体里的淋巴！）and their impact on the SRQs need to be quantified. By studying the source and nature of these constituents, they can be characterized and the necessary next steps to reduce them can be identified.

In a modeling and simulation environment, every source of uncertainty has a varying degree of impact on the overall uncertainty.理解关系 Then, they can be addressed by a specific way depending on their nature. If they are present because of a lack of knowledge about it（认知问题） (i.e., epistemic uncertainty), can by definition be reduced [10]. The means to achieve this goal can be through designing a study or experiment that would generate new information about the model or the parameters in question. In this paper, the focus will be on how to design a targeted experiment for uncertainty reduction purposes（说要提供新的how！）. Such experiments are not be a replication of the same experimental setup in a more trusted domain, but a new setup that is tailored specifically for generating new knowledge pertaining to that source of uncertainty.

减少无知的影响！

An important consideration is in pursuing targeted experiments is the allowed time and budget of the program. If a lower-level, targeted experiment to reduce uncertainty is too costly or carries even more inherent unknowns due to its experimental setup, it might be undesirable to pursue by the designers. Therefore, these lower-level experiments must be analyzed on a case-by-case basis, and the viability need to be assessed. There will be a trade-off on how much reduction in uncertainty can be expected, against the cost of designing and conducting a tailored experiment. From a realistic perspective, only a limited number of them can be pursued. Considering the number of simulations used in the process of design of a new aircraft, trying to validate the accuracy of every parameter or assumption of every tool will lead to an insurmountable number of experiments. For the ultimate goal of reducing the overall uncertainty, the sources of uncertainty that have the greatest impact on the quantities of interest must be identified.工程的价值是实现目的，不是科学，大概其把问题搞清楚，受控就可以了！ Some parameters that have relatively low uncertainty may have a great effect on a response whereas another parameter with great uncertainty may have little to no effect on the response. 这是公差的另外一个说法

In summary, the train of thought that leads to experimentation to reduce the epistemic uncertainty in modeling and simulation environments is illustrated in Figure 2. If the physics of the problem are relatively well understood, then a computational model can be developed. 理解清楚物理关系的数学模型算是基础！If not, one needs to perform discovery experiments, simply to learn about the physics phenomenon 解决认知的问题！ [11]. Then, if the results of this model are consistent and accurate, then it can be applied to the desired problem. 仿真结果跟测试结果一致！ If not, aforementioned lower-level experiments can be pursued to reduce the uncertainty in the models. Created knowledge should enable the reduction of uncertainty in the parameters, or the models; reducing the overall uncertainty.理解清楚关系，才能控制不确定性！

1.3Reduced-Scale Experimentation，航空航天行业的缩小模型制样测试——也是一种预测！

An ideal, accurate physical test would normally require the building duplicates of the system of interest in the corresponding domain so that obtained results reflect actual conditions. Although this poses little to no issues for computational experiments -barring inherent modeling assumptions-, as the scale of the simulation artifact does not matter for a computational tool, it has major implications on the design of ground and flight tests. Producing a full-scale replica of the actual product with its all required details for a certain simulation is a expensive and difficult in the aerospace industry. Although full-scale testing is necessary for some cases and reliability/certification tests [12], it is desirable to reduce the number of them. Therefore, engineers always tried to duplicate the full-scale test conditions on a reduced-scale model of the test artifact.

The principles of similitude have been laid out by Buckingham in early 20th century [13, 14]. By expanding on Rayleigh’s method of dimensional analysis[15, 16], he proposed the Buckingham-Pi Theorem. This method enables to express a complex physical equation in terms of dimensionless and independent quantities ($\mathrm{\Pi }$ groups). Although they need not to be unique, they are independent and form a complete set. These non-dimensional quantities are used to establish similitude between models of different scales. When a reduced scale model satisfies certain similitude conditions with the full-scale model, the same response is expected between the two. Similitudes can be categorized in three different groups[17]:

1. 1. 

   Geometric similitude: Geometry is equally scaled

2. 2. 

   Kinematic similitude: “Homologous particles lie at homologous points at homologous times" [18]

3. 3. 

   Dynamic similitude: homologous forces act on homologous parts or points of the system.

1.4Identification of Critical Uncertainties via Sensitivity Analysis算是不错的概念

Over the recent decades, the variety of problems of interest has led to the development of many sensitivity analysis techniques. While some of them are quantitative and model-free [19], some depend on the specific type of the mathematical model used [20]. Similar to how engineering problems can be addressed with many different mathematical approaches, sensitivity analyses can be carried out in different ways. For the most basic and low-dimensional cases, even graphical representations such as scatter plots may yield useful information about the sensitivities [21]. As the system gets more complicated however, methods such as local sensitivity analyses (LSA), global sensitivity analyses (GSA) and regression-based tools such as prediction profilers may be used.

筛选过滤关键参数！——类似工程图的关键尺寸、重要尺寸！

LSA methods provide a local assessment of the sensitivity of the outputs to changes in the inputs, and are only valid near the current operating conditions of the system. GSA methods are designed to address the limitations of LSA methods by providing a more comprehensive assessment of the sensitivity of the outputs to changes in the inputs. Generally speaking, GSA methods take into account the behavior of the system over a large range of input values, and provide a quantitative measure of the relative importance of different inputs in the system. In addition, GSA methods do not require assumptions about the relationship between the inputs and outputs, such as the function being differentiable, and are well suited for high-dimensional problems. Therefore, GSA methods has been dubbed the golden standard in sensitivity analysis, in the presence of uncertainty [22].

Variance-based methods apportion the variance caused in the model outputs with respect to the model input and their interactions. One of the most popular variance-based methods is Sobol Method [23]. Consider a function $f(X)=Y$, Sobol index for a variable $X^i$ is the ratio of the variability obtained by calculating variability due to all other inputs to the overall variability in the output. These variations can be obtained from parametric or non-parametric sampling techniques. Following this definition, the first-order effect index of input $X^i$ can be defined as:

$$S_1^i=\frac{V_{X^i}(E_{X^{-i}}(Y\mid X^i))}{V(Y)}$$

(1)

where the denominator represents the total variability in the response $Y$ whereas the numerator represents the variation of $Y$ while changing $X^i$ but keeping all the other variables constant. The first-order effect represents the variability caused by $X^i$ only. Following the same logic, combined effect of two variables $X^i$ and $X^j$ can be calculated:

$$S_1^i+S_1^j+S_2^{ij}=\frac{V_{X^{ij}}(E_{X^{-ij}}(Y\mid X^{ij}))}{V(Y)}$$

(2)

Finally, since the total effect of a variable will be the sum of all first-order effects and the sum of all interactions of all orders with other input variables. Because the sum of all sensitivity indices must be unity, then the total effect index of $X^i$ can be calculated as [24]:

$$S_T^i=1-\frac{V_{X^{-i}}(E_{X^i}(Y\mid X^{-i}))}{V(Y)}$$

(3)

Because Sobol method is tool agnostic and can be used without any approximations of the objective function, it will be employed throughout this paper for sensitivity analysis purposes.

3Demonstration and Results

3.1Formulating the Problem Ontology

Development of a next-generation air vehicle platform involves significant uncertainties. To demonstrate how the methodology would apply on such a scenario, the problem selected is aerostructures analyses for a Blended-Wing-Body (BWB) aircraft in the conceptual design stage. The goal is to increase confidence in the predictions of the aircraft range by reducing the associated uncertainty on parameters used in design, 飞机行业本质上是作坊式or我们所谓做样式，它的challenge有不一样的地方，但是有技术含量的！. A representative OpenVSP drawing of the BWB aircraft used in this work is given in Figure 6.

3.2Identification of Critical Uncertainties

For the given use case, two tools are found to be appropriate for the early-stage design exploration purposes, FLOPS and OpenAeroStruct.

As a low fidelity tool——极简模型！, NASA’s Flight Optimization System (FLOPS)[27] will be used to calculate the range of the BWB concept for different designs. FLOPS is employed due to its efficiency in early-stage design assessment, providing a quick and broad analysis under varying conditions with minimal computational resources. FLOPS facilitates the exploration of a wide range of design spaces by rapidly estimating performance metrics, which is crucial during the conceptual design phase where multiple design iterations are evaluated for feasibility and performance optimization.目标上、方法论类似ANSYS rmxprt ，它可能是降低单元自由度或者模型简化得更多一点！FLOPS uses historical data and simplified equations 那就还是跟rmxprt一样的！ to estimate the mission performance and the weights breakdown of an aircraft. Because it is mainly based on simpler equations, its run time for a single case is very low, making it possible to run a relatively high number of cases. Because FLOPS uses lumped parameters, it is only logical to go to a slightly higher fidelity tool that is appropriate with the conceptual design phase in order to break down the lumps. Therefore, a more detailed analysis of the epistemic uncertainty variables will be possible.

OpenAeroStruct [28] is a lightweight, open-source tool designed for integrated aerostructural analysis and optimization. It combines aerodynamic and structural analysis capabilities within a gradient-based optimization framework, enabling efficient design of aircraft structures. The tool supports various analyses, including wing deformation effects on aerodynamics and the optimization of wing shape and structure to meet specific design objectives. In this work it will be used as a step following FLOPS anaylses, at it represents a step increase in fidelity. According to the selected analysis tools, the main parameter uncertainties involved are to be investigated are the material properties (e.g., Young’s modulus) and the aerodynamic properties (e.g., lift and drag coefficients).

Variable Name	Description
WENG	Engine weight scaling parameter
OWFACT	Operational empty weight scaling parameter
FACT	Fuel flow scaling factor
RSPSOB	Rear spar percent chord for BWB fuselage at side of the body
RSPCHD	Rear spar percent chord for BWB at fuselage centerline
FCDI	Factor to increase or decrease lift-dependent drag coefficients
FCDO	Factor to increase or decrease lift-independent drag coefficients
FRFU	Fuselage weight (composite for BWB)
E	Span efficiency factor for wing

First, among the list of FLOPS input parameters, 31 of them are selected as they are either not related to the design, or highly abstracted parameters that may capture the highest amount of abstraction and cause variance in the outputs. Of these 31 parameters, 27 of them are scaling factors and are assigned a range between 0.95 and 1.05. Remaining four are related to the design, such as spar percent chord at BWB at fuselage and side of the body and they are swept in estimated, reasonable ranges. The parameter names and their descriptions will be explained throughout the discussion of the results as necessary, but an overview of them are listed in Table 2 for the convenience of the reader. Aircraft range is calculated through a combination of sampled input parameters. Among these parameters 4096 samples are generated using Saltelli sampling [19], a variation of fractional Latin Hypercube sampling, due to its relative ease in calculation of Sobol indices and availability of existing tools. Calculation of the indices are carried out by using the Python library SALib [29].

In Figure 7, total sensitivity indices calculated are given for three different sampling strategies. Blue bars represent the Quasi-Monte Carlo (QMC) sampling that uses all 4096 input factors. To demonstrate the impact of how sensitivity rankings may change through the use of surrogate modeling techniques, two different response equations (RSE) are employed. First one is the an RSE that is constructed using all 4096 points, and the other one is constructed using only 10% of the points, representing a degree of increase in the computational cost. After verifying that these models fit well, it is seen that they are indeed able to capture the trends albeit the ranking of important sensitivities need to be paid attention.

In this analysis, it is seen that aerodynamic properties, material properties and the wing structure location have been found to have significant impact on the wing weight and aerodynamic efficiency. Therefore, they are identified as critical uncertainties. This is indeed expected and consistent with the results found for a tube-and-wing configuration before in Reference [30].

3.3 Experiment Design and Execution

For demonstration of the methodology, the next step will include the low-fidelity aero-structures analysis tool, OpenAeroStruct [28] and how such a tool can be utilized to guide the design of a physical experimentation setup.

1. 1. 

   Problem Investigation:

   • • 

     Premise: The variations in range calculations are significantly influenced by uncertainties in the Young’s modulus, wingbox location and aerodynamic properties. These parameters are related to the disciplines of aerodynamics and structures.

   • • 

     Research Question: How can the uncertainties in these parameters impacting the range can be reduced?

   • • 

     Hypothesis: A good first-order approximation is the Breguet range equation is used to calculate the maximum range of an aircraft [31]. A more accurate determination of the probability density function describing the aerodynamic performance of the wing will reduce the uncertainty in the wing weight predictions.

2. 2. 

   Thought Experiment: Visualizing the impact of more accurately determined parameters on the simulation results, we would expect to see a reduction in the variation of the simulation outputs.

3. 3. 

   Purpose of the Experiment: This experiment aims to reduce the parameter uncertainty in our wing aero-structures model. There is little to none expected impact of unknown physics that would interfere with the simulation results at such a high level of abstraction. In other words, the phenomenological uncertainty is expected to be insignificant for this problem. In order to demonstrate the proposed methodology, both avenues —computational experiment only, and physical experiment— will be pursued.

4. 4. 

   Experiment Design: We decide to conduct a computational experiment that represents a physical experimentation setup where the parameters pertaining to the airflow, material properties and structural location are varied within their respective uncertainty bounds and observe the resulting lift-to-drag ratios. For subscale physical experiment, the boundary conditions of the experiment need to be optimized for the reduced-scale so the closest objective metrics can be obtained.

5. 5. 

   Computational Experiments for both cases:

   • • 

     Define the Model: We use the OpenAeroStruct wing structure model with a tubular spar structures approximation and a wingbox model.

   • • 

     Set Parameters: The parameters to be varied are angle of attack, Mach number, location of the structures.

   • • 

     Design the Experiment: We use a Latin Hypercube Sampling to randomly sample the parameter space. Then Sobol indices are computed to observe the global sensitivities over the input space.

   • • 

     Develop the Procedure:  （仿真跟制样测试是同步进行的！）

     • – 

       For CX only: For each random set of parameters, run the OpenAeroStruct model and record the resulting predictions, case numbers and run times. After enough runs to sufficiently explore the parameter space, analyze the results.

     • – 

       For PX only: Use the wingbox model only in OpenAeroStruct, pose the problem as a constrained optimization problem to get PX experimentation conditions, scale is now a design variable, scale dimensionless parameters accordingly.

3.4Computer Experiments for Uncertainty Mitigation

Reducing the uncertainty in the lift-to-drag ratio would have a direct impact on reducing the uncertainty in range predictions. L/D is a key aerodynamic parameter that determines the efficiency of an aircraft or vehicle in converting lift into forward motion while overcoming drag. By reducing the uncertainty in L/D, one can achieve more accurate and consistent estimates of the aircraft’s efficiency, resulting in improved range predictions with reduced variability and increased confidence.

To calculate L/D and other required metrics, the low-fidelity, open-source aerostructures analysis software OpenAeroStruct is used. BWB concept illustrated in Figure 6 is exported to OpenAeroStruct. For simplicity purposes, vertical stabilizers are ignored in the aerodynamics and structures analyses. The wingbox is abstracted as a continuous structure, spaning from 10% to 60% of the chord, throughout the whole structural grid. This setup is used for both CX and PX cases, and the model parameters are manipulated according to the problem.

3.4.1Test conditions

First, it is necessary to develop a simulation that would develop the full-scale conditions. The simpler approximation models the wingbox structure as a tubular spar, with a reducing diameter from root to tip. The diamaters are calculated through optimization loops so that stress constraints are met. For the wingbox model, the location is approximated from the conceptual design of the structrual elements from public domain knowledge. For both cases, different aerodynamic and structural grids are employed to investigate the variance in SRQs. Cruise conditions at 10000 meters are investigated, with a constant Mach number of 0.84.

Five different model structures are tested for this experimentation setup with the same set of angle of attack, Mach number, spar location and Young’s modulus in order to make an accurate comparisons. Through these runs, lift-to-drag ratios are calculated and the histogram is plotted in Figure 8. In this figure, the first observation is that although the wingbox model is a better representation of reality, its variance is higher than compared to the tubular spar models, as well as being hypersensitive to certain inputs in some conditions. It is also seen that the predictions of the tubular-spar model generally lie between the predictions that of the two different fidelities of the wingbox model.

Furthermore, the runtime statistics have a significant impact on how the results are interpreted, as well as how many cases can be realistically considered. The overview is presented in Table 3. It is obvious that the mesh size is the most dominant factor on the average run time for a single case. An interesting observation is that the coarser wingbox model takes less time to run compared to a lower fidelity tubular spar model, and predicts a higher lift-to-drag ratio. The reason of this was the wingbox model with the coarser mesh was able to converge in fewer iterations compared to the tubular-spar model. Using a shared set of geometry definition and parameters, as much as the corresponding fidelity level allows, showed that decreasing the mesh size resulted in less variance in the predicted SRQs, as expected. However, increasing the fidelity level comes with a new set of assumptions pertaining to the newly included subsystems, or physical behavior. Therefore, one cannot definitively say that increasing the fidelity level would decrease the parameter uncertainty without including the impact of the newly added parameters.

Run type	Std. Deviation in L/D	Mean Runtime [s]
Tubular spar-coarse	0.184	6.79
Tubular spar-medium	0.169	15.38
Wingbox - coarse	0.794	2.5
Wingbox - medium	0.376	20.7
Wingbox - fine	0.401	76.67

3.5Leveraging Computer Experiments for Guiding Physical Experimentation  建立缩小版本的飞机，设计物理测试方案！

3.5.1Feasibility of the full-scale model

As discussed before, construction and testing of a full-scale vehicle models is almost always not viable in the aerospace industry, especially in the earlier design phases. For this demonstration, a free-flying sub-scale test will be pursued. The baseline experiment conditions will be the same as in the computational-only experimentation, except for appropriate scaling of parameters. Therefore, a scale that would be an optimum for a selected cost function needs to be found, considering the constraints. For this use case, following constraints are defined:

• • 

  Scale of the sub-scale model: 

• • 

  Mach number: , The effects of compressibility are going to be much more dominant as $Ma=1$ is approached, therefore the upper limit for the Mach number was kept at 0.87.

• • 

  Angle of attack , Because all similitude conditions will not be met, it the flight conditions for a different angle of attack need to be simulated. This is normal practice in subscale testing [32].

• • 

  Young’s Modulus: ,Young’s modulus of the model, should be less than three times of the full-scale design.

3.5.2Optimize for similitude

For this optimization problem, a Sequential Least Squares Programming method is used. SLSQP is a numerical optimization algorithm that is particularly suited for problems with constraints [33]. It falls under the category of sequential quadratic programming (SQP) methods, which are iterative methods used for nonlinear optimization problems. The idea behind SQP methods, including SLSQP, is to approximate the nonlinear objective function using a quadratic function and solve a sequence of quadratic optimization problems, hence the term “sequential". In each iteration of the SLSQP algorithm, a quadratic sub-problem is solved to find a search direction. Then, a line search is conducted along this direction to determine the step length. These steps are repeated until convergence. One advantage of SLSQP is that it supports both equality and inequality constraints, which makes it quite versatile in handling different types of problems. It is also efficient in terms of computational resources, which makes it a popular choice for a wide range of applications.

procedure Optimization(x)

     Define Scaling parameters

     Define x=[$n$, $\alpha$, Ma, $h$, E]

     Define constraint s

     Initialize x with initial guess [0.1, 0, 0.84, 10000, 73.1e9]

     while not converged do

         Evaluate cost function f(x)$▷$ Equation 7

         Solve for gradients and search directions

         Run OpenAeroStruct optimization

         if Failed Case then

              Return high cost function

              Select new x

         end if

     end while

     return x

end procedure

The algorithm used for this experiment is presented in Algorithm 1. For convenience purposes, the altitude is taken as a proxy for air density. In the optimization process, mass (including the fuel weight and distribution) is scaled according to:

$$n_{mass}=\frac{{\rho }_F}{{\rho }_S}{n}^3$$

(4)

where ${\rho }_F$ is the fluid density for the full-scale model, ${\rho }_S$ the fluid density for the sub-scale model, and $n$ is the geometric scaling factor [32]. Since aeroelastic bending and torsion are also of interest, following aeroelastic parameters for bending ($S_b$) and torsion ($S_t$) need also be satisfied, and they are defined as:

$$S_b=\frac{EI}{\rho V^2{L}^4}$$

(5)

$$S_t=\frac{GJ}{\rho V^2{L}^4}$$

(6)

These two parameters need to be duplicated in order to assure the similitude of inertial and aerodynamic load distributions for the same Mach number or scaled velocity, depending on the compressibility effects at the desired test regime [32]. And the cost function is selected to be:

$$f(x)={\left|\frac{{\frac{C_L}{C_D}}_S-{\frac{C_L}{C_D}}_F}{{\frac{C_L}{C_D}}_F}\right|}^2+30{\left|\frac{R{e}_S-R{e}_F}{R{e}_F}\right|}^2+3000{\left|\frac{M{a}_S-M{a}_F}{M{a}_F}\right|}^2$$

(7)

where, lift-to-drag ratio, Reynolds number and the Mach number of the sub-scale model are quadrically penalized with respect to their corresponding deviation form the simulation results of the full scale result. Because the magnitudes of the terms are vastly different, second and third terms are multiplied with coefficients that would scaled their impact to the same level of the first term. For other problems, these coefficients present flexibility for engineers. Depending on how much certain deviations in the ratio of similarity parameters are penalized, the optimum scale and experiment conditions will change. In this application, the simulated altitude for the free-flying model is changed, rather than changing the air density directly.

3.5.3Interpret results

Without the loss of generality, it can be said that the optimum solution may be unconstrained or constrained, depending on the nature and the boundaries of the constraints. At this step, the solution will need to be verified as to whether it will lead to a feasible physical experiment design. Probable causes will be conflicts between the design variables, or infeasibility to match them in a real experimentation environment. In such cases, the optimization process needs to be repeated with these constraints in mind. However, for this problem, the constrained optimum point is found to be:

• • 

  $n=0.2$

• • 

  $Ma=0.86$

• • 

  $\alpha =10$

• • 

  $E_S=219GPa$

• • 

  $h=0m$

• • 

  $Re=6.2x{10}^6$

Furthermore,, it can be noted that due to the nature of the constrained optimization problem, the altitude for the sub-scale test was found to be sea-level. While, this is not completely realistic, it points to an experiment condition where high-altitude flight is not necessary. Finally, the Young’s modulus for the sub-scale model is slightly below the upper threshold, which was three times that of the Young’s modulus of the full-scale design. With this solution we can verify that solving a constrained optimization problem to find the experiment conditions is a valid approach, and provides a baseline for other sub-scale problems as well.

Furthermore, the runtime statistics have a significant impact on how the results are interpreted, as well as how many cases can be realistically considered. The overview is presented in Table 3. It is obvious that the mesh size is the most dominant factor on the average run time for a single case. An interesting observation is that the coarser wingbox model takes less time to run compared to a lower fidelity tubular spar model, and predicts a higher lift-to-drag ratio. The reason of this was the wingbox model with the coarser mesh was able to converge in fewer iterations compared to the tubular-spar model. Using a shared set of geometry definition and parameters, as much as the corresponding fidelity level allows, showed that decreasing the mesh size resulted in less variance in the predicted SRQs, as expected. However, increasing the fidelity level comes with a new set of assumptions pertaining to the newly included subsystems, or physical behavior. Therefore, one cannot definitively say that increasing the fidelity level would decrease the parameter uncertainty without including the impact of the newly added parameters.

Table 3:OpenAeroStruct runtime and output variation statistics with respect to different model structures, on a 12-core 3.8 GHz 32GB RAM machine

3.5 Leveraging Computer Experiments for Guiding Physical Experimentation

3.5.1 Feasibility of the full-scale model

As discussed before, construction and testing of a full-scale vehicle models is almost always not viable in the aerospace industry, especially in the earlier design phases. For this demonstration, a free-flying sub-scale test will be pursued. The baseline experiment conditions will be the same as in the computational-only experimentation, except for appropriate scaling of parameters. Therefore, a scale that would be an optimum for a selected cost function needs to be found, considering the constraints. For this use case, following constraints are defined:

• Scale of the sub-scale model: n < 0.2

• Mach number: 0.8 < Ma < 0.87 (The effects of compressibility are going to be much more dominant as Ma = 1 is approached, therefore the upper limit for the Mach number was kept at 0.87)

• Angle of attack: 0 < α < 10 (Because all similitude conditions will not be met, the flight conditions for a different angle of attack need to be simulated. This is normal practice in subscale testing)

• Young's Modulus: Es < 3Ef (Young's modulus of the model should be less than three times of the full-scale design)

3.5.2 Optimize for similitude

For this optimization problem, a Sequential Least Squares Programming method is used. SLSQP is a numerical optimization algorithm that is particularly suited for problems with constraints [33]. It falls under the category of sequential quadratic programming (SQP) methods, which are iterative methods used for nonlinear optimization problems.

The idea behind SQP methods, including SLSQP, is to approximate the nonlinear objective function using a quadratic function and solve a sequence of quadratic optimization problems, hence the term "sequential". In each iteration of the SLSQP algorithm, a quadratic sub-problem is solved to find a search direction. Then, a line search is conducted along this direction to determine the step length. These steps are repeated until convergence.

One advantage of SLSQP is that it supports both equality and inequality constraints, which makes it quite versatile in handling different types of problems. It is also efficient in terms of computational resources, which makes it a popular choice for a wide range of applications.

The algorithm used for this experiment is presented in Algorithm 1. For convenience purposes, the altitude is taken as a proxy for air density. In the optimization process, mass (including the fuel weight and distribution) is scaled according to:

复制

n_mass = (ρ_F / ρ_S) * n^3

where ρF is the fluid density for the full-scale model, ρS the fluid density for the sub-scale model, and n is the geometric scaling factor [32].

Since aeroelastic bending and torsion are also of interest, following aeroelastic parameters for bending (Sb) and torsion (St) need also be satisfied, and they are defined as:

复制

Sb = (EI) / (ρ * V^2 * L^4)
St = (GJ) / (ρ * V^2 * L^4)

These two parameters need to be duplicated in order to assure the similitude of inertial and aerodynamic load distributions for the same Mach number or scaled velocity, depending on the compressibility effects at the desired test regime [32].

And the cost function is selected to be:

复制

f(x) = |((CL/CD)_S - (CL/CD)_F) / (CL/CD)_F|^2 
        + 30 * |((Re_S - Re_F) / Re_F)|^2 
        + 3000 * |((Ma_S - Ma_F) / Ma_F)|^2

where, lift-to-drag ratio, Reynolds number and the Mach number of the sub-scale model are quadrically penalized with respect to their corresponding deviation form the simulation results of the full scale result. Because the magnitudes of the terms are vastly different, second and third terms are multiplied with coefficients that would scaled their impact to the same level of the first term. For other problems, these coefficients present flexibility for engineers. Depending on how much certain deviations in the ratio of similarity parameters are penalized, the optimum scale and experiment conditions will change. In this application, the simulated altitude for the free-flying model is changed, rather than changing the air density directly.

3.5.3 Interpret results

Without the loss of generality, it can be said that the optimum solution may be unconstrained or constrained, depending on the nature and the boundaries of the constraints. At this step, the solution will need to be verified as to whether it will lead to a feasible physical experiment design. Probable causes will be conflicts between the design variables, or infeasibility to match them in a real experimentation environment. In such cases, the optimization process needs to be repeated with these constraints in mind.

However, for this problem, the constrained optimum point is found to be:

• n = 0.2

• Ma = 0.86

• α = 10

• Es = 219 GPa

• h = 0 m

• Re = 6.2 x 10^6

Furthermore, it can be noted that due to the nature of the constrained optimization problem, the altitude for the sub-scale test was found to be sea-level. While, this is not completely realistic, it points to an experiment condition where high-altitude flight is not necessary.

Finally, the Young's modulus for the sub-scale model is slightly below the upper threshold, which was three times that of the Young's modulus of the full-scale design. With this solution we can verify that solving a constrained optimization problem to find the experiment conditions is a valid approach, and provides a baseline for other sub-scale problems as well.

4 Conclusion

In this work, a novel approach is introduced for the design of physical experiments, emphasizing the quantification of uncertainty to target the of engineering models小语法错误 with a specific focus on early-stage aircraft design. Sensitivity analysis techniques are intelligently utilized to find out specific computational and physical experimentation conditions, to tackle the challenge mitigation of epistemic uncertainty.

Findings indicate that this methodology not only facilitates the identification and reduction of critical uncertainties through targeted experimentation but also optimizes the design of physical experiments through computational efforts.作者可能对数据库的理解不够！感觉它不懂SQL,不然可以走得更远一点！ This synergy enables more precise predictions and efficient resource utilization. Through a case study on a Blended-Wing-Body (BWB) aircraft concept, the practical application and advantages of the proposed framework are exemplified, demonstrating how subsequent fidelty levels can be leveraged for uncertainty mitigation purposes.

Presented framework for uncertainty management that is adaptable to various design challenges. The study highlights the importance of integrating computational models by guiding physical testing, fostering a more iterative and informed design process that will save resources. Of course, every problem and testing environment got its own challenges. Therefore, dialogue between all parties involved in model development and physical testing is encouraged.

Future research is suggested to extend the application of this methodology to different aerospace design problems, including propulsion systems and structural components. Additionally, the development of more advanced computational tools and algorithms could further refine uncertainty quantification techniques. With more detailed models and physics, integration of high performance computers, it is possible to see the impact of this methodology in later stages of the design cycle. The reduction of uncertainty on performance metrics can contribute to avoiding program risk — excessive cost, performance shortcomings and delays.

n5321 | 2025年7月11日 00:00

Legend and legacy : the story of Boeing and its people  

序言写的比较煽情。最近对波音公司感兴趣。这本书只能read online

n5321 | 2025年7月9日 22:27

Any SpaceX Engineers want to share some process advice with “old space”?

I work for what y’all would probably consider an “old space” entity and lately have been trying to figure out how to improve and sculpt our development and verification processes. Obviously SpaceX has been doing innovations in this regard, and in my opinion, is the distinguishing factor in their effectiveness. We have brilliant GN&C people, but if it takes 8 hours to run a sim when the dragon engineers have flight hardware on their desk, one of those teams is going to be able to test and improve ideas faster.

So here are some things I’ve sort of learned that I think spaceX is capitalizing on that others aren’t as much, fill in the blanks if you can.

• Using proven COTS products when possible

• When a proven COTS product is not available, build it in-house to reduce the chance of a garbage contractor burning you (we suffer from this a lot)

• A focus on modern software development practices and applying that attitude to the vehicle. SpaceX benefits from being in the field of launch vehicles, which can be tested more readily in the actual operating environment than say a Martian orbiter/lander. I’d say they probably focus more on getting compile time and sim time tightened up as much as possible as well but I don’t know that for a fact.

• Generally having less assessment time and more actually just make a decision and build it time to get it to the environment quicker. Again maybe not possible to extend as extremely to things outside of LEO but we’ll see. I know they couldn’t do this as much with dragon, which is closer to what I would want to emulate at my employer as opposed to starship.

What else am I missing? I don’t think there’s any great reason other entities can’t operate as quickly as SpaceX does, I really think eventually the methods used will spread and gain traction. Let’s speed that up a bit ;). If it makes you feel any better about spilling my employer is in no way competing with SpaceX, we only ever collaborate. Help us help you!

McLMark

I'm in "old space" right now, and I'm fairly young.

I've found so many things I'd like to change coming from an agile background. I'll keep my list short, though:

1. GD&T needs to go. If the prototype part works, it works. That's it. You can load test it, but if there's any reason you're not automatically assembling it after machining, you're doing something wrong.

2. Stop using drawings for assemblies and internally machined parts (NX Drafting). I've spent 90%+ of my time making a drawing correct when the model and notes themselves were perfect. It's just stupid. Document your work and design intent - but don't have such specific requirements for drawings that make you churn out 10x fewer components.

3. Stop holding "delegation" meetings. If you see something that needs to be done, do it. If you need someone to do something, add it to their backlog and talk to them about it. We don't need a 3 hour weekly meeting to discuss who is going to work on what for these few specific components.

4. Improve prototype manufacturing flow. If I want to test an idea, let me make a cheap (maybe scaled down) version of it without having to go through the typical release process (which takes months and multiple peer reviews).

5. Test complete assemblies more often. Continue to test individual parts, but if you put it all together and test it, you can pretty quickly find a bunch of issues - especially stuff like interferences and material property issues.

EDIT: edited point 1 to better express my frustration with GD&T for prototype parts.

GD&T needs to go. If the part works, it works. That's it. You'll load test it, but if there's any reason you're not automatically assembling it after machining, you're doing something wrong.

I hate doing GD&T as much as the next guy but this is absurd. Manufacturing processess are not infinitely precise or repeatable. Finding out the part doesn't fit isn't suddenly ok just because you test fit it immediately after machining. There are very few instances where a machined part can be immediately assembled without secondary operations anyway (is machining suddenly the only manufacturing process in existence?). And what about replacement parts? Without GD&T (or something like it), there is no guarantee the part will fit in every assembly.

You're basically advocating for reverting to a world of before interchangeable parts.

I can't talk about space but for comercial nuclear its very similar phenomenon from what I can tell.

Extreme reluctance to try new technologies. So much so people have to retire before upgrading.

Kids show up straight out of college and have to learn 1970s technologies or software. Like they could eliminate an entire wing of the building with an Audrino and 3 weeks programing but aren't allowed

Specs are treated like the Bible. If you don't know where it came from and often you don't you aren't allowed to change it. These were often just picked from a long list of suitable solutions or numbers in the 70s because the loudest person wanted it. Now risk adverse management and regulatiots won't change.

Implementing anything substantial requires you to convince your manager who then has to convince their manager on up to like 4 levels. The 4th level came from energy sales and has no respect for technical input or technical decision making. Once your 4th level management is approved they have to personally take the case to the regulator at some risk. The government regulator then takes 2 years.

Spacex apparently has few of these limitations. I don't think any amount of lower level process improvement will help until the whole system is overhauled.

n5321 | 2025年7月7日 23:46

1 Introduction

Digital thread is a data-driven architecture that links together information from all stages of the product lifecycle (e.g., early concept, design, manufacturing, operation, postlife, and retirement) for real-time querying and long-term decision-making [1,2]. Information contained in the digital thread may include sufficient representation (e.g., through numerical/categorical parameters, data structures, and textual forms) of available resources, tools, methods, processes, as well as data collected across the product lifecycle. A desired target of digital thread is its use as the primary source from which other downstream information, such as that used for design, analysis, and maintenance/operations related tasks, can be derived [2,3].

Although a significant challenge of the digital thread involves development of an efficient architecture and its associated processing of information, a relatively unexplored aspect of the digital thread is how to represent and understand the propagation of the uncertainty within the product lifecycle itself. 研发过程中最具挑战性的问题 。High levels of uncertainty can lead to designs that are overconservative or designs that may require expensive damage tolerance policies, redesigns, or retrofits if analysis cannot show sufficient component integrity. One way to reduce this uncertainty is to incorporate data-driven decisions that are informed from data collected throughout the design process.

To assess the benefits of data-driven decisions, incorporating the value of future information into the decision-making process becomes critical. Performing this type of analysis opens up the possibility to assess not only just the current product generation or iteration but also the ones to follow. From this, new ways of thinking about design emerge, such as how can strategic collection of data from the current generation be used to improve the design of the next? For example, we may start asking what data should be collected, where should the data be collected, and when should the data be collected to minimize overall accrued costs? This problem involves analyzing uncertainty and the data collected over sequential stages of decision-making. In this article, we showcase how our digital thread analysis methodology, introduced in Ref. [4] and further developed in Ref. [5], analyzes this problem using Bayesian inference and decision theory, and solves it numerically using approximate dynamic programming.

To set the stage for discussion, we give a brief overview of the aspects of digital thread that are of relevance. Digital thread can be seen as a synthesis and maturing of ideas from product lifecycle management (PLM), model-based engineering (MBE), and model-based systems engineering (MBSE). PLM is the combination of strategies, methods, tools, and processes to manage and control all aspects of the product lifecycle across multiple products [6]. These aspects might include integrating and communicating processes, data, and systems to various groups across the product lifecycle. A key enabler of efficient PLM has been the development and implementation of MBE where data models or domain models communicate design intent to avoid document-based exchange of information [7,8], the latter which can result in lossy transfer of the original sources, as well as to eliminate redundant and inconsistent forms of data representation and transfer. Examples of MBE data models include the use of mechanical/electronic computer-aided design tools and modeling languages such as system modeling language (SysML), unified modeling language, and extensible markup language (xml). MBSE applies the principles of MBE to support system engineering requirements related to formalization of methods, tools, modeling languages, and best practices used in the design, analysis, communication, verification, and validation of large-scale and complex interdisciplinary systems throughout their lifecycles [8–11]. Many recent assessments and applications of these ideas in the context of the digital thread can be found in additive manufacturing [12], 3D printing and scanning [13], computer numerical control machining [14], as well as detailed design representation, lifecycle evaluation, and maintenance/operations related tasks [15,16].

With a functional digital thread in place, characterizing uncertainty and optimizing under it can be performed with statistical methods and techniques. For instance, relevant sources of uncertainty within the product lifecycle, such as design parameters, modeling error, and measurement noise, can be identified, characterized, and ultimately reduced using tools and methods from uncertainty quantification [17–19]. With a means of assessing uncertainty, optimization under uncertainty can be performed with stochastic-based design and optimization methods. For instance, minimizing probabilistically formulated cost metrics subject to constraints that will arise for the digital thread decision problem may involve utilizing either stochastic programming or robust optimization. In stochastic programming, uncertainty is represented with probabilistic models, and optimization is performed on an objective statement with constraints involving some mean, variance, or other probabilistic criteria [20]. Alternatively, in robust optimization, the stochastic optimization problem is cast into a deterministic one through determining the maximum/minimum bounds of the sources of uncertainty and performing an optimization over the range of these bounds [21]. In addition, if reliability and robustness at the system level is required, uncertainty-based multidisciplinary design optimization can be employed [22–24].

Of course, decision-making using the digital thread is not a one-time occurrence. Understanding the sequential nature of the multistage decision problem of the digital thread where one decision affects the next is critical for producing effective data-driven decisions. These decisions will have to be guided through some appropriate metric of assessing costs and benefits. This problem is explored in optimal experimental design where the objective is to determine experimental designs (in our case decisions) that are optimal with respect to some statistical criteria or utility function [25]. To assess the sequential nature of decision-making for the digital thread in particular, sequential optimal experimental design can be employed where experiments (again, decisions in our case) are conducted in sequence, and the results of one experiment may affect the design of subsequent experiments [26].

Despite the range of development and growth of digital thread and its application to manufacturing, maintenance/operations, and design related tasks in various multidisciplinary settings, a principled formulation that considers the propagation of uncertainty in the product lifecycle in the context of the digital thread is sparse. Furthermore, a mathematically precise way of analyzing and optimizing sequential data-informed decisions as new information is added to the digital thread remains absent. To address these gaps, in this article, we show that (1) the digital thread can be considered as a state that can dynamically change based on the decisions we make and the data we collect. We then show that (2) the evolution of uncertainty within the product lifecycle can be described with a Bayesian filter that can be represented in the digital thread itself. After expressing the digital thread in this way, we show how (3) the evolution of the digital thread can be modeled as a dynamical system that can be controlled using a stochastic optimal control formulation expressed as a dynamic program where the objective is to minimize total accrued costs over multiple stages of decision-making. Finally, we provide a (4) numerical algorithm to solve this dynamic program using approximate dynamic programming.

We illustrate our methodology through an example composite fiber-steered component design problem where the objective is to minimize total accrued costs over two design generations or iterations. In addition to evaluating design choices such as performing coupon level experiments to reduce uncertainty in materials as well as manufacturing and deploying a component to obtain operational data, effectiveness of data collection can be further tailored by determining where to place sensors (sensor placement) or selecting which sensors to use (sensor selection). The novelty in our approach is that these choices will be guided by the objective directly without the need for additional metrics or criteria.

The rest of this article is organized as follows, Sec. 2 sets up the illustrative design problem through which our methodology will be described as well as lays out the mathematical machinery that describes the dynamical process underlying the digital thread. Section 3 presents the decision problem for the digital thread-enabled design process and presents the numerical algorithm to solve the decision problem. Section 4 presents the results for the example design problem. Finally, Sec. 5 gives concluding remarks.

2 Design Problem Formulation

In this section, we formulate the design problem to be solved using our methodology. Section 2.1 describes the example design scenario through which we convey our methodology, Sec. 2.2 lays down the mathematical description of the problem, and Sec. 2.3 describes the underlying dynamical models that will be used for the overall decision-making process.

2.1 Scenario Description.

The design problem involves finding the optimal fiber angle and component thickness for a composite tow-steered (fiber-steered) planar (2D) component subject to cost and constraint metrics. We consider specifically the design of a chord-wise rib within a wingbox section of a small fixed wing aircraft of wingspan around 15 m, as shown in Fig. 1. The overall geometry has five holes of various radii with curved top and bottom edges.

Fig. 1

View largeDownload slide

Geometry, initial sensor locations, and boundary conditions for the design problem for one loading condition. Transverse shear is directed out of the page and is not shown for clarity.

A challenge to our design task is the presence of uncertain inputs that directly influence the design of the component. In this problem, the uncertain inputs are the loading the component will experience in operation, the material properties of the component, and the specific manufacturing timestamps. Situations where these variables have most relevance occur during the early stage of design when testing and experimentation have not yet taken place or when a brand new product is brought to market where only partial information can be used from other sources due to its novelty.

Large uncertainties in these inputs can lead to conservative designs that can be costly to both manufacture and operate. Thus, the goal is to collect data to reduce these uncertainties to the degree necessary to minimize overall costs. Data can be collected through three different lifecycle paths: material properties can be learned through collection of measurements from coupon level experiments; manufacturing timestamps can be learned from a combination of a bill of materials, timestamps of individual processes, and other related documentation when a prototype or product is manufactured; and operating loads can be learned from strain sensors placed on the component in operation.

Although the task of learning the uncertain input variables through measurements can be addressed with methods from machine learning, and more classically from solution methods for inverse problems, this task in the context of the overall design problem is made complicated by the fact that collecting data come at a cost. To see this, we illustrate the digital thread for this design problem in Fig. 2. Here, we see that collecting relevant data can require both time and financial resources. Although material properties data can be obtained fairly readily and quickly during the design phase through coupon level experiments, manufacturing data can only be obtained once a prototype is built. In addition, operational data can only be obtained once a prototype or a full component is built, equipped with sensors, and put into operation. Depending on the scale of the component, the manufacturing process can take weeks or months, and putting a full component into operation with proper functionality of all its parts and sensor instrumentation may take much longer. Thus, making cost-effective decisions that reduce uncertainty is critical for product reliability, reducing design process flow time, and minimizing total product expenses across its lifecycle.

Fig. 2

View largeDownload slide

Illustration of digital thread for the example design problem. The product lifecycle stages of interest here are between design and operation.

2.2 Mathematical Formulation.

The key elements of the design problem are broken down into five items: a notion of time or stage, the uncertain input variables (what we would like to learn), the measurement data (what we learn from), the digital thread itself (how to represent what we know), and the decision variables (the decisions and design choices we can make).

Time or Stage. Time is modeled using nondimensional increments that enumerate the sequence of decisions made or to be made up to some finite horizon ⁠. It is expressed as ⁠. Physical time is allowed to vary and will be the case when different decisions take shorter physical times to execute (e.g., performing coupon experiments) or longer physical times to execute (e.g., manufacturing and deploying a component).

Inputs. The N_y inputs (the uncertain quantities to be learned) are described at each stage t as ⁠. The variable y_t is composed of parameters of a finite discretization of the five integrated through thickness traction terms (in-plane loads per unit length, in-plane moments per unit length, and transverse shear per unit length) on the component boundary for a particular operating condition, material properties (material strengths), and parameters of a manufacturing process model to compute process times consisting of N_m total steps. The composite structural model is based on the small displacement Mindlin–Reissner plate formulation [27,28] specialized for composites. For the manufacturing process model, we employ the manufacturing process and associated parameters detailed in the Advanced Composite Cost Estimating Manual (ACCEM) cost model [29,30] that consists of N_m = 56 total steps for our problem.

Measurements. The N_z measurements are described at each stage t as ⁠. The variable z_t is composed of three strain sensor components for N_s sensor locations located on the top surface of the component, material properties data determined from coupon level experiments, and timestamps for the N_m total steps of the manufacturing process. Coupon level experiments here involve the static failure of composite test specimens of appropriate loading and geometry to acquire data about material properties and failure strengths used for structural analysis. Note, as illustrated in Fig. 2, the components of z_t are taken at different points along the product lifecycle (corresponding to coupon tests, manufacturing, and operation) and may not be fully populated at every stage t.

Decisions. Decision-making will encompass different strategies related to performing coupon tests and manufacturing and deploying a new design to reduce uncertainty while minimizing costs. Associated with manufacturing and deployment are additional specifications of fiber angle, component thickness, and sensor placement/selection. We designate a high-level decision as ⁠, where  will correspond to performing coupon tests and  will correspond to manufacturing and deployment. For ⁠, we designate the additional design specifications as ⁠, where  specifies the geometrical parametrization of the component and  specifies the parametrization for sensor selection.

For this problem,  is composed of the coefficients of a finite dimensional parametrization of fiber angle and through thickness of the component body, as well as the spatial locations for all N_s sensors. For simplicity, we model ply angle as a continuous function of the component body and not model more detailed specifications such as individual ply and matrix compositions, additional layers consisting of different ply types, ply thicknesses, number of plies, and ply cutoffs at boundaries.

For sensor selection, we use a soft approach based on activation probabilities [31] to avoid the combinatorial problem associated with an exact binary (on/off) representation. Specifically,  gives probabilities, where the ith sensor is selected (active or on) with probability  and not-selected (inactive or off) with probability  during operation. To quantify the effective number of active sensors for this particular parametrization, we use an effective sensor utilization quantity defined as follows:

(1)

Here,  when all sensors are always off/inactive and  when all sensors are always on/active.

Digital Thread. The digital thread  at stage t reconciles the uncertain parts of the product lifecycle with its certain (or deterministically known) parts as follows:

(2)

where R_t is the representation of the deterministically known parts of the product lifecycle at stage t that we will collectively call resources,  is the representation of the uncertainty within the product lifecycle itself at stage t, and  is an information space over the product lifecycle encapsulating all possible uncertain and certain elements across all stages ⁠. Provided a criterion of sufficiency is maintained [5], the digital thread can be represented in a number of equivalent ways. In this article, the uncertainty within the product lifecycle is represented using  that specifies the probability distribution of the uncertain inputs y_t given the history of collected data I_t = {R₀, u₀, …, u_t−1, z₀, …, z_t−1} and the current decision to be made u_t. The resources R_t are represented using a multi-data type set that contains numerical, categorical, and/or character-like specifications of:

1. Methods, Tools, and Processes: Enterprise level information and protocols of available methods, tools, and processes across the product lifecycle.

2. Products: Product-specific design geometry, manufacturing process details, operational and data collection protocols, operation/maintenance/repair history, and lifecycle status.

2.3 Dynamical Process of the Digital Thread.

With the design problem modeled, the dynamics of the digital thread can be described using the transition model

(3)

where  evolves the digital thread from stage t to t + 1 given the decision u_t and measurements z_t at stage t. Within this transition model,  is updated using the Bayesian filter

(4)

while the resources are updated according to

(5)

Here, the integration is performed over the uncertain inputs y_t and ν is the measure (or volume) over the uncertain inputs y_t. The function Ψ_t allows for changing resources (adding new elements, updating existing elements, or removing existing elements) at stage t. The Bayesian filter in Eq. (4) models the process of data assimilation from stage t to stage t + 1. First, the likelihood term p(z_t|y_t, R_t, u_t) inside the integral represents the collection of new measurements after a decision is performed, followed by the updating of our knowledge of y_t after incorporating those measurements. Next, the term p(y_t+1|y_t, R_t+1, u_t+1) represents inheritance and modification of information carried over from a previous design into the next design (e.g., loads from a past airplane reused and modified for a new airplane with a longer fuselage). Details of the derivation of this Bayesian filter can be found in Ref. [5].

The Bayesian filter in Eq. (4) can be computed using sequential Monte Carlo methods [32] or other linear/nonlinear filtering methods where appropriate. For linear models with Gaussian uncertainty, for example, the Bayesian filter is a variant of the Kalman filter (with the prediction and analyze steps reversed) and can be computed analytically. The resources can be managed and updated through MBE, MBSE, and PLM related tools or software as well as through other data management techniques.

3 Decision-Making Using the Digital Thread

In this section, we describe the decision-making problem for the design problem. Section 3.1 describes the specific decisions of interest, Sec. 3.2 describes the mathematical optimization problem associated to those decisions, Sec. 3.3 describes the approximate dynamic programming technique we employ to solve the mathematical optimization, and finally Sec. 3.4 provides a numerical algorithm to implement the approximate dynamic programming technique.

3.1 Decisions for the Problem Scenario.

For this problem scenario, we will produce two generations of a component where we are allowed to perform one set of coupon experiments. This will correspond to a three-stage problem where ⁠. In particular, we will be interested in the high-level decision sequences  that we will denote as EDD and  that we will denote as DED. For example, the sequence DED means to manufacture and deploy a new design first, followed by performing coupon level experiments second, and finally manufacturing and deploying another new design. The second design benefits from data collected from both coupon experiments and operational measurements of the previous design. These two sequences are distinguished by whether coupon experiments should be performed before any design is ever manufactured (and subsequently deployed) via the sequence EDD or right after the first design is manufactured and deployed via the sequence DED.

For each of these two sequences, we are interested in how subsequent data assimilation influences designs and costs of the component over the two generations. This will be explored through a greedy scenario that makes no use of future information, a sensor placement scenario, and a sensor selection scenario. Sensor placement and selection are not combined together for this problem setup to assess the performance of each (location versus activity) independently.

3.2 Decision Statement for the Digital Thread.

The decision statement for the digital thread-enabled design process is given by the following Bellman equation:

(6)

Here,  is the optimal value function or cost-to-go at stage t, the parameter γ ∈ [0, 1] is a discount factor, and the symbol * denotes optimal quantities or functions. The solution to this Bellman equation yields an optimal policy  that defines a sequence of functions  specifying new designs and changes to the Digital Thread for each stage t up to the horizon T. Each function  of the optimal policy is a function of  (a feedback policy), i.e., ⁠. The expectation is taken over the uncertain inputs {y_t, …, y_T} and measurements {z_t, …, z_T}.

The functions  and  denote the stage-wise cost and constraint functions, respectively, for the problem with N_g total constraints. For the stage-wise cost model, we employ a linear combination of the manufacturing cost model with cost functions that penalize aggressive fiber angle variation, aggressive component thickness variation, operational costs including strain sensor usage, as well as costs associated to coupon tests [5]. We do not penalize the placement of sensors for this particular setup. For the stage-wise constraint function during design stages, we use the Tsai-Wu failure criterion [33].

In some cases, the distribution of g_t may be heavily tailed in which case the expected value of the constraint given in Eq. (6) may not produce robust enough designs. In those cases, the expected value of the constraint can be replaced with an appropriate measure of probabilistic risk, e.g.,  for some small ɛ > 0, or other criteria that can also take into consideration the severity of failure associated to the value of g_t [34].

For the greedy scenario, the Bellman equation is solved by setting γ = 0. Although data assimilation still occurs through evolution of the digital thread via the transition model from stage t to t + 1, decisions determined from the Bellman equation at stage t with γ = 0 do not take into consideration the benefits nor costs of future data assimilation because the value of future information that comes through V_t+1 as stage t is canceled out. For the sensor placement scenario, γ = 1 and the sensor locations are allowed to vary while the sensor selection probabilities are all fixed at one. For sensor selection, γ = 1 and the sensor selection probabilities are allowed to vary while the sensor locations are fixed. Note that structural tailoring for the sensor selection and sensor placement setups also takes place by design of the Bellman equation because the future value function V_t+1 is a function of u_t. This dependency enables fiber angle and component thickness to also control the effectiveness of subsequent data collection in addition to sensor placement or sensor selection.

In total, there are six policies to compare: the greedy, sensor placement, and sensor selection scenarios for both the EDD and DED high-level decision sequences.

3.3 Solving the Decision Problem Using Approximate Dynamic Programming.

We solve the decision problem by first rewriting the Bellman equation given in Eq. (6) to produce the following equivalent, but notationally simpler statement:

(7)

where

(8)

The function  is the expected value of the forward t + 1 optimal value function, O_t is the expected value of the stage-wise cost function at stage t, and G_t is the expected value of the stage-wise constraint function at stage t. Next, we use a combination of Monte Carlo sampling with policy and function approximation [35] to solve the optimization problem numerically. The functions O_t and G_t can be computed directly using Monte Carlo sampling methods. The remaining terms that need to be determined, namely, ⁠, ⁠, and ⁠, will be approximated and updated using policy and function approximation. However, to apply the methods of function approximation to our problem, ⁠, ⁠, and  need to first have explicit parametrized forms. The parametrized forms we use are as follows:

(9)

where  is a vector of M_p basis functions at stage t,  is a matrix of basis coefficients for the policy function at stage t,  is a matrix of basis coefficients for the value function at stage t,  is a vector of M_v basis functions at stage t, and  is a matrix of basis coefficients for the expected value of the forward value function at stage t. The variables  and  are used for setting initial conditions (e.g., initial component fiber angle, component thickness, and sensor locations).

The symbol * is dropped because we are now approximating the optimal functions with imposed structure, which may lose optimality. In addition, the expression for V_t and S_t is exponentiated to ensure nonnegativity of the value function. For the implementation, the basis functions ϕ_t and φ_t are constructed using radial basis functions (Gaussian radial basis functions in particular) with inputs appropriately scaled to lie within the bi-unit hypercube of appropriate dimension. These basis functions are not direct functions of the digital thread, but functions of numerical features of the digital thread such as mean, variances, and/or or other statistical metrics of  as well as other relevant numerical quantities from R_t.

Next, A_t, B_t, and C_t are trained using a combination of least squares and approximate solutions of the Bellman equation. However, a direct application of least squares is challenging because we need to have a means of generating samples to train A_t, B_t, and C_t in the first place. We also do not know in advance how many samples are sufficient to yield good estimates of the policy and value functions, so we would like to have flexibility of incorporating new samples without much recalculation of the least squares formulas. Furthermore, performing the inverses in these least squares formulas can become computationally expensive for large dimensions of A_t, B_t, and C_t. Finally, we would like to have an incremental update rule to the approximation of the policy where new samples are obtained through exploration using the latest approximation of the policy.

These issues can be addressed by utilizing a recursive update of the least squares formulas, known as recursive least squares (RLS) [36]. Associated with the RLS formulation is a class of recursive approximate dynamic programming techniques given in Refs. [37] and [38] that we parallel. Details of the derivations for the RLS equations used in this article can be found in Ref. [5]. The RLS equations take the form:

(10)

Here, j is the update index that increases by one when a new data point is added, ⁠, and  at some digital thread  and decision ⁠. The terms  and  are determined from the minimization:

(11)

while  is given by

(12)

The decision  is generated by sampling in the neighborhood of the current iteration of the policy or decision iterates during optimization of Eq. (11). Note, this decision is different than  to allow additional flexibility on when and where to update S_t. This is because unlike μ_t and V_t, S_t is a function of both the digital thread and decision and thus requires a different sampling approach to capture different values of the decision at a given state of the digital thread.

The symmetric matrix  at j = 0 is the regularization term in the original least squares formulas for A_t and B_t. It is updated using the Sherman–Morrison matrix identity:

(13)

An identical formula holds for the symmetric matrix  by swapping out  with  and  with  in Eq. (13).

The digital thread  at stage t + 1 is determined through forward simulation with the latest iteration of the policy ⁠:

(14)

while the digital thread  at stage t = 0 is sampled from ⁠. Sampling the digital thread  at stage t = 0 involves sampling different distributions for ⁠, which can be performed through using a suitable hyperprior distribution or through direct sampling of the parameters of the parametrization of ⁠, if applicable. Sampling R₀ involves direct sampling from the set of allowable values (discrete and/or continuous) its elements can take.

3.4 Numerical Implementation to Solve the Multistage Decision Problem.

The numerical implementation for the algorithm presented in Sec. 3.3 is divided between Algorithms 1–3. To initialize and train a policy, first initializePolicy is called and then trainPolicy is called however many times is necessary until a convergence threshold on the policy or value function is achieved [38]. Details of the subroutines are given in the following.

Simulate and Initialize Policy. Simulating a policy is described in simulatePolicy. Here, a given policy π₀ along with a digital thread  at stage  are used to generate the future evolution of  from stage s onward. The output is the trajectory (i.e., states) of the digital thread for t ∈ {s, …, T}. Note, the transition model outputs the digital thread from stage t + 1 from stage t, so stage T of the digital thread is computed from stage T − 1, thus the for loop is truncated to stage T − 1.

Information about product lifecycle elements (statistics of inputs, resources, products in operation, and their digital twins) at some t ∈ {s, …, T} within this trajectory is extracted through postprocessing of the appropriate ⁠. Measurements are synthetically generated if no physical measurements are available during offline training. Online measurements come directly from test data or the actual physical systems.

To initialize a policy from scratch, initializePolicy is called taking in as input the high-level decision sequence and initial condition parameters. Here, the parameters η_t, κ_t > 0 represent the scaling factors of the initial least squares regularization term (in this implementation, the identity matrix of appropriate size).

Simulate and initialize policy

Algorithm 1

1: proceduresimulatePolicy(⁠⁠)

2:  fordo

3:   

    ⊳Obtain measurement through forward simulation or from physical system

4:   

5:   

6:  return

1: procedureinitializePolicy

2:  fordo

    ⊳Initialize values at each stage t

3:   

4:   

5:   

6:    with 

7:    with 

    ⊳Update parameters ofwith initialized values

8:   

9: 

10: return

Train policy

Algorithm 2

1: proceduretrainPolicy(⁠⁠)

  ⊳Build a skeleton trajectory to perform updates

2:  Sample 

3:  simulatePolicy(⁠⁠)

   ⊳Update terms going backwards from

4:  for do

5:   

    ⊳Assign cost and constraint functions for deterministic optimizer at stage

6:   costFunction(⁠⁠)

7:   constraintFunction(⁠⁠)

    ⊳Run deterministic optimizer

8:   optimizer(⁠⁠)

    ⊳Updateandusing RLS update rule

9:   

10:   

11:   

12:   

13:   

14: return